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ABSTRACT 


This  thesis  excunined  the  impact  of  docximent  design, 
organization,  and  writing  style  of  a  Report  of  Investigation 
(ROD  on  the  adjudicators  charged  with  assessing  them. 
Specifically,  the  adjudicators'  assessment  time,  impression  of 
case  information,  perception  of  the  field  agent  writing  the 
cases,  and  writing  style  preferences  were  analyzed.  A  total 
of  (40)  Army,  Navy,  DISCR,  and  Air  force  adjudicators  were 
tested  using  both  "high"  and  "low-impact"  style  ROI's.  A 
four  part  questionnaire  captured  their  assessment  times, 
impressions,  perceptions,  and  preferences  between  the  two 
different  style  cases. 

The  results  show  that  the  two  style  cases  produce 
statistically  different  adjuaicator  impressions  of  case 
information,  and  perceptions  of  the  field  agents  writing  the 
cases.  They  also  show  that  adjudicators  prefer  the  high- 
impact  style  over  the  low-impact  ones.  However,  the  high- 
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I .  INTRODUCTIOM 

Over  the  last  several  years,  the  Department  of  Defense 
(DoD)  has  sought  to  improve  the  personnel  security  screening 
process.  This  desire  was  prompted  primarily  by  the  rash  of 
security  breaches  over  the  last  several  years.  One  area  where 
improvement  is  desired  is  the  area  of  information  assessment. 
This  assessment  occurs  when  personnel  at  adjudication 
facilities  evaluate  Personnel  Security  Investigation  (PSI) 
documentation  obtained  from  the  Defense  Investigative  Services 
(DIS) . 

The  Report  of  Investigation  (ROD  is  the  primary 
information  conduit  between  these  agencies.  More 
specifically,  the  ROI  is  the  written  interface  between  the  DIS 
Field  Agent  who  conducted  the  PSI  and  the  adjudicator 
responsible  for  assessing  the  case. 

One  objective  of  all  involved  in  the  security  screening 
process  is  to  improve  the  quality  and  clarity  of  the  ROI.  ROI 
improvements  will  hopefully  increase  reader  comprehension  and 
general  doctiment  effectiveness.  Several  important  factors 
that  affect  the  quality  of  the  ROI  are  the  docioment  design, 
organization,  and  writing  style. 

Two  studies  (Suchan  1989,  and  1992)  have  examined  these 
factors  for  the  Defense  Personnel  Security  Research  and 
Education  Center  (PERSEREC) .  Suchan' s  research  is  outlined  in 
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the  PERSEREC  technical  report  titled  "Comparison  of  High  and 
Low-impact  Report  of  Investigations  on  Adjudicators'  Decision 
Making:  A  Pilot  Study".  His  primary  focus  in  that  study  is  to 
determine  the  affect  that  organization,  docviment  design,  and 
stylistic  factors  have  on  the  decision  making  ability  of 
adjudicators.  This  thesis  builds  on  that  work.  It 
specifically  examines  how  the  ROI  document  design, 
organization  and  writing  style  affect  the  following: 

•  The  impressions  of  the  adjudicators  concerning  the  case 
information; 

•  The  perception  of  the  adjudicators  about  the  field  agent 
who  wrote  the  case; 

•  The  preference  of  the  adjudicators  for  one  of  the  two  case 
styles  used  in  the  study; 

•  The  case  processing  time  required  by  the  adjudicators. 

The  remainder  of  this  chapter  will  discuss  in  more  detail 
the  background  of  Suchan's  research,  present  the  specific 
research  question  that  the  thesis  will  attempt  to  answer,  and 
then  end  with  a  preview  of  the  remaining  chapters. 

A.  BACKGROUND 

As  Stated  earlier,  this  thesis  continues  the  ongoing 
research  of  Suchan.  His  research  is  concerned  with  the 
adjudicators  ability  to  accurately  read  ROI's  and  make 
reliable  and  consistent  judgements  about  a  subject's  security 
worthiness.  Suchan's  research  objective  is  to  determine  if 
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changes  in  document  design,  organization,  and  writing  style, 
will  enable  adjudicators  to  make  higher  quality  decisions. 
This  decision  making  ability  of  the  adjudicator  is  the  crux  of 
the  personnel  security  process,  which  has  the  purpose  of 
weeding -out  subjects  who  are  capable  of  committing  espionage - 

Drawing  from  the  existing  body  of  literature  that  defines 
the  qualities  of  a  clearly  written  document,  Suchan  composited 
those  qualities  into  two  ROI's.  The  clearly  written  ROI's 
contained  organizational  features  such  as  internal  previews 
and  clear  transitional  tags;  docviment  design  characteristics 
such  as  strategic  use  of  lists,  bullets,  heading  and  white - 
space;  and  stylistic  traits  such  as  active  verbs  and  right - 
branching  modifiers.  The  total  of  these  features  is  commonly 
referred  to  as  a  "high- impact"  style. 

The  converse  of  the  high- impact  style  is  referred  to  as 
the  "low-impact"  style.  This  is  the  status-quo  style  field 
agents  currently  use  when  writing  ROI's.  This  style  includes 
features  such  as  poor  document  design,  "wall-to-wall"  prose, 
no  headings  or  lists,  and  no  use  of  bold  print  or  other 
highlighting  devices;  lack  of  internal  previews  at  the 
beginning  of  long  sections;  long  convoluted  sentences  that 
strain  short-term  memory;  passive  verbs;  and  finally  lack  of 
old  inf ormation/new  information  patterns  to  create  coherence. 

Suchan  tested  both  the  high- impact  and  low-impact  ROI's  on 
a  sample  of  (40)  adjudicators  from  four  adjudication  sites; 
(12)  from  the  Army  Central  Adjudication  Facility;  (12)  the 
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Navy  Central  Adjudication  Facility;  (4)  from  the  Air  Force 
Security  Clearance  Office;  and  (12)  from  the  Directorate  for 
Industrial  Security  Clearance  Review  at  Columbus,  Ohio.  His 
objective  was  to  determine  whether  differences  in  ROI  design, 
organization,  and  writing  style  affected  the  decision  making 
ability  of  the  tested  adjudicators. 

A  four  part  questionnaire  was  used  to  collect  the 
necessary  data  for  the  research,  although  Suchan  only  analyzed 
the  first  part  of  that  questionnaire.  The  remaining  three 
parts  of  the  questionnaire  collected  data  concerning  the 
following; 

•  The  impressions  of  the  adjudicators  concerning  the  case 
information; 

•  The  perception  of  the  adjudicators  about  the  field  agent 
who  wrote  the  case; 

•  The  preference  of  the  adjudicators  for  one  of  the  two  case 
styles  used  in  the  study; 

Suchan' s  research  results  provide  the  basis  for  this 
thesis.  The  unexeunined  data  from  the  remaining  three  parts  of 
the  questionnaire  were  used  as  the  primary  source  of  data  for 
the  analysis.  The  results  of  his  study  are  outlined  in  the 
next  chapter,  the  Literature  Review,  while  a  more  detailed 
description  of  his  research  design  is  included  in  the  Research 
Methodology  Chapter. 
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B.  RESEARCH  QUESTIONS 

This  thesis  will  answer  the  following  primary  and 
secondary  research  question: 

1.  Primary  Research  Questions 

This  thesis's  primary  objective  is  to  analyze  the 
remaining  three  parts  of  the  Suchan  questionnaire. 
Specifically,  it  will  determine  if  there  is  a  statistical 
significance  in  the  questionnaire  responses  for  ROI's  written 
in  a  high- impact  style  versus  a  low-impact  style.  The 

primary  research  questions  for  the  thesis  are  as  follows: 


•  Will  there  be  statistically  significant  differences 

between  adjudicators'  in^ressions  of  their  ability  to 
process  case  information  presented  in  a  high- impact  ROI 
versus  a  low-impact  one? 

•  Will  there  be  statistically  significant  differences 

between  adjudicators'  perceptions  or  feeling  towards  the 
field  agents  who  wrote  a  high- impact  ROI  as  con^ared  to 
those  who  wrote  the  low-iitpact  ROI? 

•  Will  there  be  statistically  significant  differences  in 
adjudicators'  preferences  for  the  case  information 
presented  in  a  high- impact  versus  the  low-impact  style? 

•  Will  there  be  statistically  significant  differences  in  the 
adjudicators'  processing  time  of  the  case  information 
presented  in  a  high- impact  style  versus  the  low-impact 
style? 

2.  Secondazy  Research  Questions 

The  thesis  will  answered  the  following  secondary 
research  questions: 

•  Will  there  be  any  statistical  differences  in  the  primary 
research  questions  presented  above,  when  results  from  the 
adjudicators  at  the  various  central  adjudication  sites  are 
analyzed  independently? 
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•  Will  there  be  any  statistical  differences  in  the  primary 

research  questions  presented  above,  when  the  population  of 

adjudicators  is  broken  down  demographically? 

C .  CHAPTER  PREVIEW 

This  section  discusses  the  organization  of  the  remainder 
of  the  thesis  and  previews  each  coming  chapter. 

1.  Literature  Review 

An  outline  of  the  results  of  prior  related  research 
and  a  discussion  of  the  significant  published  literature  on 
the  subject  of  document  design,  organization,  and  writing 
style  will  be  presented, 

2.  Research  Methodology 

A  detailed  description  of  the  research  methodology 
used  in  the  thesis  is  presented.  Both  case  selection  and 
questionnaire  content  are  exanples  of  topics  discussed  in  this 
chapter. 

3.  Presentation  and  Analysis  of  Adjudication  Times  and 

the  Adjudicator's  la^resslons  of  Case  Information 

A  presentation  and  analysis  of  the  adjudication  times 
and  Part  II  of  the  questionnaire  is  the  focus  of  the  chapter. 
Part  II  covers  the  irtpressions  of  the  adjudicators  concerning 
the  case  information, 

4.  Presentation  and  Analysis  of  Questionnaire  Part  III 

Results:  the  Adjudicator's  Perceptions  of  Field  Agents 

Similar  to  the  Preceding  chapter,  an  analysis  of  Part 
III  of  the  questionnaire  and  discussion  of  the  statistical 
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testing  results  is  presented.  Part  III  tests  the  perception 
of  the  adjudicators  about  the  field  agents  who  wrote  the  case. 

5.  Presentation  and  Axialysis  of  Questionnaire  Part  IV 
Results:  the  Writing  Style  Preferences  of  Adjudicators 
This  chapter  present  the  result  of  Part  III  of  the 

Questionnaire.  Part  III  determine  the  adjudicators 
preferences  between  the  high  and  low-impact  writing  style. 

6.  Conclusions  2md  Recommendations 

This  chapter  summarizes  the  conclusions  and 
recommendations  of  the  thesis. 
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II. 


LITERATURE  REVIEW 


A.  CHAPTER  INTRODUCTION 

This  chapter  focuses  on  the  significant  published 
literature  concerning  the  subject  of  document  design, 
organization,  and  writing  style.  Much  of  the  document 
improvement  strategies  discussed  in  this  literature  was 
incorporated  into  the  revised  high- impact  ROI's.  The 
literature  review  will  be  presented  in  three  sections.  The 
first  section  outlines  the  published  body  of  literature 
associated  with  the  doctiment  factor  stated  above;  the  second 
section  shows  the  limited  amount  of  literature  available  in 
the  following  areas: 

•  Reader  impressions  of  written  communication, 

•  Reader  perception  of  the  writer  of  a  written 
communication, 

•  Reader's  writing  style  preferences; 

and  finally,  the  third  section  gives  a  detailed  description  of 
the  results  of  several  PERSEREC  studies  in  the  area  of  ROI 
document  design,  organization  and  writing  style.  Included  in 
this  third  section,  will  be  a  discussion  of  the  results  of  the 
Suchan  study  on  which  this  thesis  is  a  continuation. 
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B.  DOCOMENT  FACTORS 

1.  ReadeJsillty  Formulas 

Readability  formulas  are  important  because  for  decades 
researchers  have  used  them  as  a  dominant  tool  for  measuring 
document  effectiveness.  This  study  avoided  using  these 
formulas  to  determine  the  relative  readcdsility  and  efficiency 
of  current  ROI's  because  of  their  significant  limitations. 
One  particular  researcher  (Seltzer,  1981)  summarizes  their 
limitation  the  best. 

Seltzer  argues  that  because  readability  formulas  only 
assess  sentence  length  and  word  choice,  they  fail  to  account 
for  important  reader  conprehension  factors.  These  factors 
include : 

•  the  reader's  interest  in  the  document; 

•  the  reader's  familiarity  with  a  specialized  language  that 
might  be  used  in  a  document; 

•  the  effect  that  convoluted  sentences,  poorly  placed 
modifiers,  and  other  syntactic  difficulties  have  on  a 
reader's  ability  to  process  a  sentence; 

•  the  reader's  ability  to  see  easily  logical  association 
between  groups  of  sentences  and  paragraphs. 

Another  important  research  study  (Hirsch,  1977)  points 
out  other  flaws  in  these  formulas.  Hirsch  argues  that 
readability  formulas  fail  to  consider  several  in^ortant 
factors  that  contribute  to  a  document's  effectiveness.  He 
believes  that  to  make  a  document  readcJsle  and  easy  to 
understand,  the  writer  must  use  organizational,  syntactic,  and 
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linguistic  strategies  that  will  constrain  meaning.  This  is 
accomplished  by  reducing  the  semantic -syntactic  possibilities 
that  can  occur  in  a  clause,  sentence,  paragraph,  or  even  a 
document.  Hirsch  also  contends  that  subject -verb  patterns, 
the  voice  of  verbs,  clause  length,  and  transitional  tags  can 
be  used  in  conjunction  with  each  other  to  iit^rove  document 
processing  speed  and  comprehension. 

It  is  clear  that  there  are  severe  limitations  to  using 
readability  formulas  when  assessing  the  effectiveness  of  a 
document.  The  above  two  researchers  point  out  their 
significant  flaws  and  their  argument  against  these  formulas. 
However,  they  only  allude  to  the  specific  factors  that  make  a 
document  effective. 

Those  specific  factors  include  subject-verb-object ; 
transitional  tags  and  internal  previews;  active  verbs; 
shortened  paragraphs,  sentence  and  clause  length;  and  several 
others.  These  factors  were  used  to  generate  the  revised  high- 
impact  document  in  the  Suchan  Study.  The  remainder  of  this 
section  will  be  devoted  to  outlining  the  literature  that 
describes  the  above  stated  dociiment  effectiveness  factors. 

2 .  Document  Organization 

Several  business  communication  researchers  have  made 
significant  discoveries  about  the  factors  that  influence 
information  processing.  Two  particular  researchers  (Fielden 
&  Dulek,  1984)  have  studied  and  assessed  document 
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effectiveness  from  a  macro -organizational  perspective.  They 
argue  that  a  "bottom  line"  communication  technique  decreases 
reading  time  and  improves  document  comprehension.  The  essence 
of  this  technique  is  for  the  writer  of  nonsensitive  documents 
to  state  their  purpose  for  writing  in  the  first  paragraph. 

These  researchers  further  argue  that  to  decrease 
reading  time  and  improve  comprehension  in  longer  documents, 
the  writer  should  also  include  a  contract  sentence  immediately 
following  the  bottom  line.  The  contract  sentence,  basically 
an  internal  preview,  organizes  the  remainder  of  the 
communication  for  the  reader  by  providing  a  skeletal  framework 
of  the  major  point  to  be  covered  and  the  order  in  which  they 
will  be  presented. 

Although  Fielden  and  Dulek  cite  no  empirical  evidence 
to  support  their  claims  about  the  advantages  of  bottom  lines 
and  contract  sentences,  the  work  of  two  cognitive 
psychologists  (Bransford  &  Johnson,  1972  and  1973)  justify  the 
value  of  these  two  organizational  strategies.  Bransford  and 
Johnson  found  that  subjects  who  were  provided  a  concrete  title 
to  a  very  general,  context-free,  written  passage  could  better 
understand  and  remember  the  passage  than  those  not  provided  a 
title.  The  title  provided  a  mental  model  that  helped  readers 
classify  and  integrate  information.  Similarly,  a  bottom  line 
and  contract  sentence  may  help  a  reader  create  a  mental  model 
of  a  document's  content  and  constrain  its  meaning. 
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Another  study  supports  Pielden's  and  Dulek's  findings. 
This  study  (Suchan,  1989)  assessed  the  coinmunication 
efficiency  of  the  high- impact  style  versus  the  traditional 
bureaucratic  low-impact  style  favored  by  the  government.  The 
study  involving  262  naval  officers,  and  Suchan  found  that  the 
high- impact  style  took  the  officers  between  17%  and  23%  less 
time  to  read  than  the  bureaucratic  style,  and  produced  much 
better  message  comprehension. 

3 .  Informational  Design 

Another  document  factor  that  determines  its 
effectiveness  is  visual  design.  Visual  design  includes  the 
writers  use  of  headings,  list,  bold  print,  underscoring, 
spacing,  and  so  on.  Several  researchers  (Kostelnick,  1988; 
Hartley  &  Trueman,  1985;  Battison  &  Landesman,  1981)  have 
confirmed  that  using  these  strategies  iiiproves  effectiveness. 

Kostelnick  found  that  local  design  strategies  (lists, 
bullets,  indentation,  and  so  on)  may  restrain  meaning  by 
enabling  readers  to  code  and  chunk  information  more  easily. 
Hartley  and  Trueman  have  proved  through  extensive  testing  that 
headings  improve  reader  performance.  Battison  and  Landesman 
have  shown  that  FCC  regulations  revised  into  a  more  visually 
appealing  question- answer  format  improved  reader 
comprehension . 
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4.  Sentemce  Level  Research 


Three  researchers  (Miller,  1956  and  1970;  Sever,  1972; 
Clark  &  Clark,  1968)  have  studied  how  readers  use  short  and 
long  term  memory  to  process  sentences.  In  particular,  Clark 
and  Clark  found  that  because  a  reader's  short-term  memory  can 
hold  about  seven  chunks  of  information,  readers  confronted 
with  too  much  modifying  detail  (left  embedding  of 
information) ,  may  have  difficulty  retaining  in  short-term 
memory  the  primary  subject -verb  unit  of  the  sentence. 

The  relevance  of  this  research  is  that  it  justifies 
the  need  for  writers  to  create  word  sequences  that  are  easy 
for  readers  to  form  into  stable  groups.  A  writer  should  use 
the  subject -verb- object  order  throughout  the  document  to 
create  a  sta±>le,  predictable  pattern  that  places  little  demand 
on  the  reader's  short-term  memory.  The  use  of  these  sequences 
should  increase  reading  speed  and  iirqprove  comprehension. 

5.  Active -Passive  and  Word  Choice  Research 

Three  studies  (Olson  &  Filby,  1972;  Danks  &  Sorce, 
1973;  Charrow  &  Charrow,  1979)  have  shown  that  readers  process 
active  verbs  faster  than  passive  ones.  This  is  because 
passive  verbs  invert  the  typical  subject -verb -object  order 
syntactic  pattern. 

Finally,  research  (Rosch,  1973  and  1975)  has  shown 
that  we  are  able  to  process  quickly  and  accurately  language 
that  represents  basic  classifications  or  prototypes.  Rosch 
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found  that  language  which  is  relatively  concrete  (e.g.  "cow" 
versus  "bovine")  can  trigger  other  clarifying  word  (abstract 
background  information)  stored  in  long-term  memory.  What 
constitutes  concrete  language,  however,  may  be 
organizationally  unique. 

For  example,  adjudicators  are  accustomed  to  reading 
specialized  terms  such  as  "PSI"  and  "ROI".  These  terms 
trigger  a  range  of  concrete  associations  in  the  adjudicator 
but  not  in  a  reader  unfamiliar  with  security  issues.  In 
short,  the  consistent  use  of  context -based,  organizationally 
specific  specialized  terms  does  not  degrade  comprehension. 

These  results  support  the  assertion  that  if  a  writer 
does  not  use  concrete  language  as  defined  by  the 
organizational  context,  the  reader  may  not  see  recognizable 
prototypes  and  thus  must  attempt  to  construct  them  while 
reading.  This  construction  process  increases  reading  time, 
strains  short- term  memory,  reduces  comprehension,  and  degrades 
document  effectiveness. 

C.  READER  PERCEPTION,  IMPRESSION,  PREFERENCE  RESEARCH 

An  exhaustive  on-line  computer  search  was  conducted  to 
determine  the  extent  of  prior  empirical  research  on  readers 
perception  of  writers  based  on  writing  style  and  document 
design.  Only  3  studies  were  located. 

Brown  and  Herndl  (1986)  found  that  upper-level  managers  in 
a  large  engineering  firm  perceived  reports  written  in  a 
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bureaucratic  style  to  be  more  competent  and  objective  than 
reports  using  plain  language  guidelines  that  the  company  had 
recently  implemented.  Rogers  (1989)  discovered  that  automotive 
managers  strongly  resisted  using  company  plain  language 
criteria  in  their  dealer  contact  reports  because  they  believed 
they  would  be  perceived  by  their  bosses  as  being  less  thorough 
and  analytical. 

Another  Study  (McClure,  1990)  surveyed  the  lower 
management  levels  of  Sprint/United  Telephone  of  Florida  to 
determine  if  these  managers  perceived  their  reading 
preferences  to  be  the  same  as  those  of  their  upper- level 
managers.  McClure  found  that  employees  were  more  aware  of 
their  boss's  reading  preferences  than  their  CEO's.  Also,  as 
communication  flowed  to  higher  levels  of  management,  the 
employee  wrote  longer  sentences,  used  bigger  words,  and  relied 
on  the  passive  voice  in  order  to  appear  more  formal, 
respectful,  and  educated.  These  choices  led  to  a  increased 
reading  difficulty. 

D.  PERSEREC  STUDIES 

As  Shown  in  the  earlier  sections  of  this  chapter,  there 
exists  a  significant  body  of  literature  that  defines  the 
qualities  of  a  clearly  written  document.  PERSEREC  has 
conducted  three  studies  (Haag,  Schroyer  &  Crawford,  1989; 
Suchan,  1989  and  1992)  that  relate  to  adjudicators'  ability  to 
process  and  assess  information. 
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1.  Haag,  Schroyer,  and  Crawford  Study 

Haag,  Schroyer,  and  Crawford  found  that  adjudicators 
are  not  satisfied  with  the  way  information  is  presented  and 
that  they  desired  the  reports  be  written  more  clearly.  They 
also  concluded  that  adjudicators  had  difficulty  locating  and 
extracting  information  from  narrative  segments  of  ROI's.  The 
adjudicators  hoped  that  the  ROI  could  be  formatted  to  parallel 
and  support  the  adjudication  decision-making  process. 

2.  Suchan  (1989)  Study 

Similarly,  Suchan  also  found  that  the  adjudicators 
were  dissatisfied  with  the  writing  style  of  the  ROI.  In  his 
other  1989  study,  he  used  talk-aloud  protocols  and  field 
interviews  to  discover  that  adjudicators  had  difficulty 
sorting  and  classifying  information  in  the  narrative  sections 
of  ROI's  with  significant  derogatory  information. 
Adjudicators  often  had  to  reread  the  report  sections,  take 
extensive  notes,  and  use  other  strategies  to  create  the 
information  patterns  necessary  for  them  to  apply  the 
adjudication  criteria.  This  1989  study  was  the  precursor  to 
the  1992  Suchan  study  on  which  this  thesis  is  based. 

3.  Suchan  (1992)  Study 

Suchan' s  primary  objective  in,  "The  Comparison  of  High 
and  Low-Impact  Report  of  Investigation  on  the  Adjudicators' 
Decision  Making:  A  Pilot  Study",  was  to  determine  whether 
adjudicators  assess  ROI's  written  in  a  high- impact  style 
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differently  from  the  same  ROI's  written  in  the  current  low- 
impact  style.  Secondly,  he  examined  if  adjudicators  at 
various  central  adjudication  sites  assessed  differently  the 
high  and  low  impact  ROI's.  Furthermore,  he  determined  if 
adjudication  site  affected  the  degree  to  which  adjudicators' 
decisions  conformed  to  those  of  their  superiors. 

Suchan's  results  contradict  previous  laboratory 
research  that  show  clear  writing,  as  exemplified  by  the  high- 
impact  style,  helps  readers  to  more  effectively  process 
documents.  The  data  show  there  are  no  statistically 
significant  differences  in  decision  type  between  the  high  and 
low  impact  ROI's.  Also,  high- impact  ROI's  did  not  result  in 
decisions  that  more  closely  matched  those  of  supervisory 
adjudicators,  except  at  the  Army  adjudication  site.  In  short, 
the  high-impact  ROI's  did  not  help  adjudicators  make  better 
quality  decisions. 


\ 
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III.  RESEARCH  METHODOLOGY 


A.  CHAPTER  INTRODUCTION 

This  chapter  gives  a  detailed  outline  of  the  research 
methodology  developed  and  used  to  gather  data  for  this  study. 
It  also  describes  the  techniques  used  by  the  author  to  analyze 
the  remaining  three  parts  of  the  questionnaire.  The  topics 
included  in  this  chapter  are  as  follows:  case  selection, 
content  and  treatment;  description  of  the  adjudicator  sample 
population;  testing  procedures;  questionnaire  content; 
statistical  methods  for  analyzing  the  questionnaire  results; 
organization  of  analysis;  and  the  limitations  of  the 
methodology  and  analysis. 

B.  CASE  SELECTION,  CONTENT,  AND  TREATMENT 
1.  Case  Selection 

Earlier  PERSEREC  research  (Wiskoff  and  Fitz,  1991) 
found  that  adjudicators  dealt  with  an  average  of  1.7  issues 
per  case  and  assessed  cases  with  multiple  issues  46.3%  of  the 
time.  Consequently,  cases  that  contained  two  issues  were  used 
in  this  study. 

Furthermore,  research  shows  that  adjudicators  most 
often  dealt  with  Financial,  Drugs /Alcohol,  and 
Emotional /Mental  issues.  One  of  the  most  frequent  double  issue 
combinations  was  drug  and  alcohol  (Lewis,  Koucheravy,&  Carney, 
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1989;  Wiskoff  &  Fitz,  1991). 

With  this  in  mind,  multiple  issue  cases  containing 
combinations  of  Financial,  Drugs /Al cohol ,  or  Emotional /Mental 
derogatory  information  were  requested  from  the  Personnel 
Investigation  Center  (PIC) .  After  careful  reading  of  (56) 
micro- fiche  cases  that  met  these  criteria,  two  cases  were 
chosen  that  best  met  the  multiple  issue  and  issue  frequency 
criteria.  These  cases  also  reflected  the  document  design, 
organizational,  and  stylistic  characteristics  typical  of  ROI's 
containing  derogatory  information  (Suchan,  1989) . 

2 .  Case  Content 

The  two  cases  selected  are  called  Czarnek  and  Rokitka, 
the  pseudonyms  for  the  subject  in  each  case.  A  brief  overview 
of  each  case  is  described  below. 

a.  Czamek 

Czarnek  is  a  24  year-old  male  applying  for  a  Top 
Secret  clearance  required  for  a  security  guard  position.  The 
major  derogatory  information  in  his  case  includes: 

•  falsification  of  his  PSQ, 

•  aberrant  behavior  (lying,  personal  threats,  and 
insubordination)  resulting  in  an  honorable  discharge  from 
the  military  after  only  5  weeks  in  service, 

•  personal  and  on-the-job  dishonesty, 

•  minor  criminal  conduct  resulting  in  loss  of  eit^loyment, 

•  minor  financial  matters 
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b.  Rokitka 


Rokitka  is  a  49  year-old  female  applying  for  a  Top 
Secret  clearance  which  is  necessary  for  her  to  work  as  a 
computer  programmer.  The  major  derogatory  information  in  her 
case  includes: 

•  alcohol/drug  abuse:  use  of  marijuana,  cocaine,  and 
barbiturates 

•  criminal  conduct:  use  of  cocaine  on  the  job 

•  falsification  of  PSQ 


3 .  Case  Treatment 

Suchan  revised  both  the  Czarnek  and  Rokitka  cases 
using  document  design,  organizational,  and  stylistic 
strategies  that  research  has  demonstrated  leads  to  improved 
comprehension  (Duffy,  1985;  Redish,  1989;  Seigel,  1978; 
Selzer,  1983;  Suchan,  1989).  Furthermore,  a  thorough 
literature  review  was  conducted  to  ensure  that  each  of  the 
independent  variables  used  in  revising  the  low-impact  ROI's 
improved  comprehension.  These  independent  variables  include: 

•  Paragraph  Length:  paragraphs  were  no  longer  than  3-4 
sentences;  one  sentence  paragraphs  were  used  to  emphasize 
important  information  (Felker,  Redish,  Peterson,  1985) . 

•  Headings  and  Subheadings:  major  headings  and  subheadings 
were  used  to  telegraph  major  sections  of  derogatory 
information  (Redish, 1989) . 

•  Lists  and  Bullets:  these  graphic  aids  highlighted 
significant  information,  broke  out  statistical 
information,  and  laid  out  complicated  chronological 
sequences  (Benson,  1985;  Rubens,  1986) . 

•  Internal  Previews:  at  the  beginning  of  long  derogatory 
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sections,  an  internal  preview  was  used  to  provide  a  quick 
overview  of  the  major  points  that  were  to  come  in  that 
section.  The  language  in  the  preview  mirrored  the  language 
in  the  major  headings  (Redish,  Battison,  Gold,  1985) . 

•  Clauses  in  Subject -Verb -Object  Order:  subjects,  verbs,  and 
objects  within  clauses  were  kept  as  close  to  each  other  as 
possible  to  avoid  strain  on  short-term  memory  (Fodor  & 
Garrett,  1967) . 

•  Modifier  Strings  Right  Branched:  long  strings  of  modifying 
information  were  placed  to  the  right  of  the  object  (right 
branching)  versus  before  the  subject  (left  branching) - 
Research  has  shown  that  right  branching  of  information 
enables  readers  to  read  faster  and  remember  more  easily 
information  (Haviland  &  Clark,  1974;  Miller  &  Isard, 
1964)  . 

•  Active  Verbs:  when  appropriate,  passive  verbs  were  changed 
to  active  verbs  to  make  clear  the  agent  of  a  particular 
action  (Mirel,  1988;  Selzer, 1983) . 

•  Explicit  Transitional  Tags:  clear  transitional  tags,  often 
left  out  in  narrative  report  writing,  were  added  to  make 
it  easier  for  readers  to  quickly  grasp  relationships 
between  ideas  and  sections  of  reports  (Guillemette,  1987)  . 


Only  material  field  agents  had  written  was  revised. 
Information  contained  in  subjects'  and  references'  statements, 
medical  assessments,  and  military  reports  remained  as 
originally  written.  Also,  two  experienced  field  agents 
reviewed  the  revised  cases  and  noted  that  though  the  cases 
"looked"  different,  there  were  no  differences  in  content. 

As  indicated  in  the  Introduction  Chapter,  the  revised 
cases  are  called  "high  impact"  and  the  original  unmodified 
cases  "low  impact." 
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C.  DESCRIPTION  OF  SAMPLE  POPULATION 


Forty  senior  adjudicators  (GS  11  through  GS  13)  from  four 
major  central  adjudication  sites  participated  in  the  study. 
Adjudication  supervisors  at  each  site  chose  the  adjudicators 
to  be  tested.  Once  again,  the  breakdown  of  adjudicators  from 
each  site  is  listed  below: 

•  Army  Central  Adjudication  Facility  (12) 

•  Navy  Central  Adjudication  Facility  (12) 

•  Air  Force  Security  Clearance  Office  (AFSCO)  (4) 

•  Directorate  for  Industrial  Security  Clearance  Review 
(DISCR)  at  Columbus  (12) 

A  high  caseload  backup  prevented  AFSCO  from  allowing  more 
adjudicators  to  participate  the  study. 

A  total  of  31  female  and  9  male  adjudicators  read  the 
cases.  Average  age  was  45,  and  average  years  of  adjudication 
experience  was  13.  The  typical  adjudicator  had  spent  slightly 
more  than  6  years  at  the  adjudication  site. 

D.  TESTING  PROCEDURES 

The  adjudicators  were  tested  on  site.  At  each  site  they 
were  divided  randomly  into  two  groups,  with  each  group 
adjudicating  two  cases. 

•  Group  I  (six  adjudicators  at  each  site  except  for  AFSCO) 
received  a  revised,  high  impact  treatment  of  the  Czamek 
case  and  an  original,  low  irr5)act  treatment  of  the  Rokitka 
case. 

•  Group  II  received  a  low  intact  treatment  of  the  Czamek 
case  and  a  high  inpact  treatment  of  the  Rokitka  case. 
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A  counterbalanced  2X2  design  was  used  to  ensure  that 
each  adjudicator  responded  to  a  different  case  written  in  the 
typical,  low-impact  style  and  the  revised  high- impact  style 
and  that  order  of  presentation  would  not  influence  results. 

At  each  adjudication  site,  the  adjudicators  assessed  the 
cases  in  a  large  room.  To  guarantee  that  all  adjudicators  were 
given  the  same  instructions,  the  researcher  read  a  prepared 
script  that  explained  the  purpose  of  the  exercise,  the 
materials  they  were  receiving,  and  process  they  were  to  use  in 
assessing  the  cases  and  completing  the  questionnaires. 


E.  COMTENT  OF  QXTESTIQNNAZRE 

Adjudicators  were  asked  to  con^lete  a  questionnaire  after 
adjudicating  each  case.  The  first  questionnaire  contained 
three  parts: 

•  Part  I  contained  seven  open-ended  questions,  requesting 
information  ed^out  the  adjudication  decision,  the  rationale 
for  the  decision,  the  adjudication  criteria  applied,  the 
mitigating  criteria  applied,  the  need  for  more 
information,  and  the  perceived  need  to  send  a  Letter  of 
Intent  (LOI) . 

•  Part  II  contained  23  questions  about  the  readability  of 
the  case,  the  ease  in  finding  derogatory  information,  the 
quality  of  field  agents'  writing,  and  so  on.  A  five-point 
bipolar  scale  was  used  to  capture  these  perceptions.  See 
Appendix  A. 

•  Part  III  examined  adjudicators'  perceptions  or  feelings 
toward  the  field  agents  who  wrote  the  ROI.  Fifteen 
semantic  differential  items  on  a  five-point  scale  were 
used  to  capture  this  data.  See  Appendix  B. 

The  second  questionnaire,  completed  after  the  second  case 

was  adjudicated,  was  identical  to  the  first  except  it 
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contained  an  additional  fourth  part  (see  Appendix  C)  that 
asked  adjudicators  which  case  was  easier  to  read,  required 
less  rereading,  increased  confidence  in  the  competence  of 
field  agents,  and  so  on.  There  were  8  case  preference 
questions  of  this  type  and  two  yes/no  questions. 

This  thesis  focuses  on  the  results  from  Part  II,  III,  and 
IV  of  the  questionnaire. 

F.  STATISTICAL  TESTING  OF  QX7ESTI0NNAIRE  RESULTS 

1.  Software  Statistical  Package 

The  questionnaire  results  from  the  Suchan  Study  were 
recorded  for  analysis  in  statistical  package  called  APL.  This 
package  was  used  to  analyze  the  results  from  Part  I  of  the 
questionnaire.  The  analysis  of  the  remaining  parts  of  the 
questionnaire  were  completed  using  the  statistical  package 
SPSS.  Much  care  was  taken  when  transferring  the  questionnaire 
result  from  APL  to  SPSS.  The  first  part  of  the  survey  was 
re -analyzed  using  SPSS  and  conpared  with  the  previous  APL 
results.  Since  the  result  were  the  same,  the  integrity  of  the 
data  was  maintained. 

2.  Confidence  Interval 

Since  this  is  a  pilot  study  of  how  document  design, 
organization,  and  writing  style  intact  upon  the  inqpressions, 
perceptions,  and  preferences  of  adjudicators,  a  5%  confidence 
interval  was  determined  as  significant  during  statistical 
analysis  of  both  the  high  and  low-impact  Czamek  and  Rokitka 
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Cases.  A  10%  confidence  interval  was  deemed  sufficient  when 


the  cases  were  analyzed  independently,  or  the  population  of 
the  adjudicators  was  stratified.  With  this  in  mind,  the  10% 
confidence  interval  results  from  our  analysis  need  to  be 
interpreted  with  care. 

3 .  Statistical  Techniques 
a.  ANOVA'a 

When  analyzing  results  of  Part  II  and  III  of  the 
questionnaire,  ANOVA's  were  used  for  determining  the 
statistical  significance  of  the  comparative  data  from  the 
high- impact  and  low-impact  reports.  The  dependent  variables 
were  the  specific  factors  coded  into  the  questions  themselves, 
while  the  independent  variable  in  all  tests  was  the  specific 
report  writing  style  {i.e.  high-irtpact  or  low-impact)  .  An 
example  of  a  dependent  variable  from  Part  II  would  be  ease  of 
reading  the  subject  interview  section. 

Jb.  Cbi-Sqaarea 

When  analyzing  results  of  Part  IV  of  the 
questionnaire,  Chi-Squares  were  used  for  determining  the 
statistical  significance  of  the  comparative  data  from 
respondents  reading  the  high- impact  and  low-impact  reports. 
Once  again,  the  dependent  variedsles  were  the  specific  factors 
coded  into  the  questions  themselves,  while  the  independent 
variedJle  in  all  tests,  was  the  specific  report  writing  style 
(i.e.  high- impact  or  low-impact) . 
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Of  the  ten  questions  in  Part  IV,  eight  were 
analyzed  using  this  technique.  Questions  6  and  8  were  not  of 
comparative  nature  (i.e.  choosing  between  Case  I  or  Case  II), 
but  were  yes/no  questions.  Because  of  this  yes/no  design, 
there  was  no  way  to  tie  these  questions  to  the  independent 
variable,  which  was  the  effect  of  the  writing  style. 


6.  ORGANIZATION  OF  ANALYSIS 

The  analysis  of  each  part  of  the  questionnaire  was  to 
determine  the  statistical  significance  between  the  question 
responses  for  the  high- impact  versus  the  low-impact  ROI's.  The 
)cey  difference  throughout  the  analysis  was  the  stratification 
of  the  adjudicator  population  and/or  the  specific  case 
treatment  analyzed.  This  breakdown  is  as  follows: 

•  Both  Cases  s  The  high/low  impact  Czarnek  and  Rokitka 
high/low-impact  cases  were  analyzed  using  the  entire 
population  of  adjudicators. 

•  Czamek  Casas :  The  high/ low -impact  Czarnek  cases  were 
analyzed  using  the  entire  population  of  adjudicators. 

•  Rokitka  Caaea:  The  high/low-impact  Rokitka  cases  were 
analyzed  using  the  entire  population  of  adjudicators. 

•  Army  Adjudication  Site:  The  high/low  irrpact  Czarnek  and 

Rokitka  high/low- intact  cases  were  analyzed  using  the 

result  from  just  the  Army  population  of  adjudicators. 

•  Naw  Adjudication  Site:  The  high/low  impact  Czarnek  and 

Rokitka  high/low- intact  cases  were  analyzed  using  the 

result  from  just  the  Navy  population  of  adjudicators. 

•  DISCR  Adjudication  Site:  The  high/ low  impact  Czarnek  and 

Rokitka  high/low- inpact  cases  were  analyzed  using  the 

result  from  just  the  DISCR  population  of  adjudicators. 
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•  n«mr><jT»»phlc  Breakdown:  The  high/low  inpact  Czamek  and 

Rokitka  high/low-impact  cases  were  analyzed  using  a 
demographic  breakdown  of  the  adjudicator  population.  A 
more  detailed  description  of  the  demographic  breakdown  is 
given  below. 


H.  DESCRIPTION  OF  DEMOGRAPHIC  BREAKDOWN 

The  questionnaires  were  analyzed  using  a  demographic 
breakdown  of  the  adjudicator  population.  The  following  four 
demographic  categories  were  used  for  this  analysis: 

•  Age;  The  first  demographic  category  was  age,  young  and 

old.  Young  was  considered  below  one  standard  deviation 

from  the  mean  adjudicator  age,  while  old  was  everyone 
else.  The  mean  age  of  adjudicators  was  44.2  years,  one 
standard  deviation  was  10.4  years,  so  the  cut-off  for  the 
young  category,  was  33.8  years  old.  Under  this  criteria, 
three  (3)  adjudicators  were  fell  into  the  young  category, 
while  (36)  fell  into  the  old. 

•  Sex;  The  second  category  was  sex,  male  and  female.  The 
sexual  breakdown  was  nine  (9)  male  to  (31)  female 
adjudicators . 

•  Grade  Level;  The  third  category  was  grade  level.  The 
population  was  stratified  between  GS-11  versus  GS-12  and 
GS-13  adjudicators.  Under  this  criteria,  nine  (9)  fell 
into  the  GS-11  category,  (28)  into  the  GS-12  and  GS-13 
category,  and  three  (3)  adjudicators  not  listing  their 
grade  level  on  the  questionnaire. 

•  Years  In  Security  Ad«t<Ti<«ytratlon  Service;  The  final 
category  was  the  number  of  years  of  service  that  a 
adjudicator  had  worked  in  security  administration.  The 
population  was  stratified  between  three  years  or  less 
experience,  versus  greater  that  three  years  experience. 
Under  this  criteria,  three  (3)  adjudicator  fell  into  the 
prior  category  and  (37)  into  the  latter. 
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I .  LIMITATIONS 


The  sample  size  of  some  of  the  adjudicator  population 
breakdowns  limited  the  strength  of  the  analysis. 
Specifically,  the  cell  sizes  were  (n=6)  when  we  compared  case 
treatment  and  decision  type  within  adjudication  sites,  and  as 
small  as  {n=3)  for  two  demographic  population  breakdowns. 
Consequently,  the  data  in  these  report  sections  must  be 
interpreted  with  care. 

Also,  Suchan's  Study  tested  the  adjudicators  in  a  quiet 
room,  free  from  distractions.  This  environment  does  not 
replicate  actual  workplace  conditions.  Adjudicators  work  in 
busy  offices  where  numerous  distractions  and  frequent 
interruptions  can  make  reading  long  ROI's  difficult.  Because 
of  this  atypical  testing  environment,  the  questionnaires 
results  do  not  capture  all  of  the  high- impact  style's  possible 
reading  ease  and  information  retention  advantages. 
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IV.  PRESENTATION  AND  ANALYSIS  OF  ADJUDICATION  TIMES  AND 
THE  ADJUDICATOR'S  IMPRESSIONS  OF  CASE  INFORMATION 

This  chapter  analyzes  both  the  times  adjudicators  required 
to  assess  the  high  and  low-impact  cases  as  well  as  their 
responses  to  Questionnaire  Part  II  (see  Appendix  A)  .  The 
chapter  is  organized  in  three  sections;  the  first  two  sections 
present  and  analyze  the  adjudication  times  and  Questionnaire 
Part  II  respectively,  while  the  final  section  draws 
conclusions . 

A.  PRESENTATION  AND  ANALYSIS  OF  THE  TIME  REQUIRED  TO 

ADJUDICATE  BOTH  THE  HIGH  AND  LOW>CASES 

These  results  indicate  that  there  was  no  statistically 
significant  differences  between  the  time  required  to 
adjudicate  the  high- impact  Czarnek  and  Rokitka  cases  versus 
their  low-impact  counterparts.  The  mean  times  to  adjudicate 
both  the  high  and  low-impact  cases  are  18.33  and  20.08  minutes 
respectively.  The  corresponding  standard  deviations  are  6.51 
and  8.54  minutes.  The  resulting  value  of  the  F-statistic  is 
1.046,  which  shows  a  significance  of  31%.  This  significance 
level  is  well  outside  of  the  5-10%  criteria. 

The  analysis  results  from  the  Czamek  and  Rokitka  cases, 
each  adjudication  site,  and  most  of  the  demographic 
breakdowns,  also  produced  no  statistically  significant 
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differences,  with  one  exception:  the  male  adjudicator 
population. 

The  male  adjudicator  results,  however,  indicate  that  there 
was  a  statistically  significant  difference  between  the  time 
they  required  to  adjudicate  the  high- impact  Czarnek  and 
Rokitka  cases  versus  their  low-impact  counterparts.  The  mean 
times  to  adjudicate  both  the  high  and  low-impact  cases  are 
13.5  and  21.75  minutes  respectively.  The  corresponding 
standard  deviations  are  1.91  and  12.2  minutes.  The  resulting 
value  of  the  F-statistic  is  3.98,  which  shows  a  significance 
of  6.6%  and  satisfies  the  10%  confidence  criteria. 

Overall,  adjudication  testing  time  results  indicate  that 
the  high- impact  design,  organization  and  writing  style  does 
not  significantly  decrease  the  time  required  to  adjudicate 
cases . 

B.  PRESENTATION  AND  ANALYSIS  OF  QUESTIONNAIRE  PART  II  RESULTS 

This  section  of  the  chapter  presents  and  analysis  the 
result  to  the  questionnaire  which  tested  the  adjudicators' 
impressions  of  case  information. 

As  stated  in  the  first  and  third  chapter.  Questionnaire 
Part  II  tests  adjudicators'  impression  concerning  case 
information.  Each  of  the  twenty- three  questions  in  this  part 
of  the  survey  asked  adjudicators  to  rate  a  specific  aspect 
concerning  the  presentation  of  case  information  such  as  its 
readability  and  the  ease  of  finding  derogatory  information. 
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A  five-point  bipolar  scale  was  used  to  capture  results. 
Specifically,  this  part  of  the  survey  was  designed  to 
determine  if  there  were  significant  differences  between  the 
response  to  the  questions  for  the  high  and  low-impact  cases. 

The  first  subsection  gives  direction  on  how  to  read  the 
tables  of  results,  and  the  last  draws  conclusions.  The 
remaining  middle  subsections  are  organized  as  follows: 

•  Both  Cases:  Analysis  of  both  Czarnek  and  Rokitka  high/ low 
impact  cases  using  entire  adjudicator  population. 

•  Czarnek  Cases:  Analysis  of  Czarnek  high/low  impact  cases 
using  entire  adjudicator  population. 

•  Rokitka  Cases:  Analysis  of  Rokitka  high/low  impact  cases 
using  entire  adjudicator  population. 

•  Separate  Adiudleatlon  sites:  Analysis  of  each  Adjudication 
site  (Army,  Navy,  and  DISCR)  using  both  cases. 

•  Male  Adjudicators :  Analysis  of  both  cases  using  Male 
Adjudicators. 

1.  Directions  for  reading  tables  of  results 

There  are  seven  tables  of  results  in  this  chapter. 
Each  table  has  a  title  block  and  seven  columns  of  information. 
Below  is  an  extract  of  the  title  block  and  column  heading 
with  an  explanation  of  each. 


SUMMARY  OF  QUESTIONNAIRE  PART  II  RESULTS 
CASES  AND  ADJUDICATORS  TESTED  (n  >  sample  size) 
ADJUDICATOR'S  IMPRESSIONS  OF  CASE  INFORMATION 


QUEST 

HI 

HI  S.D. 

LI 

LI  S.D. 

F-STAT 

SIG. 

# 

MEAN 

MEAN 

Of  F 
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The  title  block  is  the  same  for  all  tables  with  the 
exception  of  the  second  line,  which  is  highlighted  above. 
This  line  states  which  cases  and  adjudicators  were  included 
for  the  analysis.  An  explanation  of  the  seven  columns 
headings  are  as  follows: 

•  QUEST  #:  The  Part  II  question  number  analyzed. 

•  TTT  MWAW-  The  mean  score  on  a  five  point  bipolar  scale  for 
the  high- impact  cases. 

•  HI  S«D. :  The  standard  deviation  of  the  high- impact  score. 

•  T.T  MTgaw-  The  mean  score  on  a  five  point  bipolar  scale  for 
the  low-impact  cases. 

•  LI  S.D« :  The  standard  deviation  of  the  low-impact  score. 

•  F-STAT:  The  value  of  the  F  Statistic  resulting  from  the 
ANOVA  testing. 

•  SIG,  of  Ft  The  significance  of  the  F  Statistic,  which  is 
the  probability  that  the  resulting  F  value  is  the  due  to 
a  random  occurrence. 

The  results  of  questions  which  fall  within  the  5% 
confidence  interval  are  shaded  while  the  result  that  meet  the 
10%  interval  are  highlighted  in  bold  type. 

2.  The  Analysis  o£  both  the  Czamek  euid  Rokltka  Case 

Results 

The  first  primary  research  question,  outlined  in  the 
first  chapter,  asks 

•  Will  there  be  statistically  significant  differences 
between  adjudicators'  impressions  of  their  cdDility  to 
process  case  information  presented  in  a  high- impact  ROI 
versus  a  low-impact  one? 

The  answer  to  that  question  is  given  in  Table  4-1.  Out  of  the 
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results  to  twenty- three  questions,  thirteen  were  significant 
within  5%  while  another  four  were  significant  within  10%. 
Those  questions  that  met  the  5%  criteria  will  be  presented  in 
the  following  subsections. 

a.  Questions  Statistically  Significant  Within  the  5% 

Confidence  Interval 

(1)  Question  1 :  Coverage  of  Issues 

This  question  asks  to  what  extent  does  the  case 
provide  good  to  poor  coverage  of  issues  to  enable  the 
adjudicator  to  make  a  high  quality  adjudication  decision.  The 
mean  score  for  the  high- impact  cases  (1.9)  falls  just  to  the 
left  of  the  somewhat  good  rating,  while  the  mean  score  (2.4) 
for  the  low-impact  cases  lies  between  the  somewhat  good  and 
neutral  rating.  These  results  indicate  that  adjudicators  feel 
that  the  high- impact  style  provides  more  thorough  coverage  of 
the  case  issues  than  the  low  impact  style. 

Since  each  case  contained  the  same  content,  the 
design,  organization,  and  style  of  the  high- impact  ROI 
affected  adjudicators'  perception  of  the  document's 
thoroughness . 

(2)  Question  2:  Presentation  of  Derogatory 

Information 

This  question  asks  if  the  way  a  field  agent 
presents  derogatory  information  in  a  case  makes  it  easy  or 
difficult  to  apply  the  adjudication  criteria  to  that 
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TABLE  4-1 


SUMMARY  OF  QUESTIONNAIRE  PART  II  RESULTS 
ANALYSIS  OF  BOTH  CZARNEK  &  ROKITKA  CASES  (n  =  40) 
ADJUDICATOR'S  IMPRESSIONS  OF  CASE  INFORMATION 


QUEST 

# 

HI 

MEAN 

HI  S.D. 

LI 

MEAN 

LI  S.D. 

F-STAT 

SIG. 
of  F 

1 

1*9 

MM 

2.4 

4.57 

0*036 

2 

1.95 

1,15 

2.4€ 

4*87 

0.031 

3 

1.93 

1.07 

2.35 

0.89 

3.78 

0.055 

4 

1.38 

0.77 

1.65 

0.86 

3.03 

0.086 

5 

— 

0*93 

a*i8 

1.06 

; ;'!4. 15^ 

0*045 

$ 

IIBiHB 

1.17 

2.58 

1*03 

4.36 

0.04 

iilllli 

2^03 

1*21 . 

2*63 

1.17 

0*012 

s 

1.93 

2.5 

0.91 

7.98 

0.006 

9 

3.4 

1.54 

3.5 

1.06 

0.081 

0.777 

10 

2.55 

1.28 

2.85 

1 

2.01 

0.16 

u 

1*9 

1*08 

1.11 

5.91 

0*017 

12 

-  m3  - 

0.96 

2.33 

0.94 

5.44 

13 

2.23 

1.39 

2.58 

1.17 

1.47 

0.229 

14 

1.9 

2.5 

5*37 

0.023 

IS  ' 

i'*oa 

2.3 

1.16 

16 

i.a 

1.02 

2.48 

1.15 

— ■ 

0.008 

17 

3.55 

1.11 

3.38 

1.13 

1.21 

0.275 

18 

3.88 

1.07 

3.45 

1.11 

3.01 

0.087 

19 

a. OS 

1*11 

2.6 

4.11 

20 

3.53 

1.04 

3.18 

1.11 

2.08 

0.153 

21 

4.68 

0.66 

4.48 

0.78 

1.57 

0.214 

22 

4.36 

0.95 

3.78 

1*25' 

5*79 

0.019 

23 

3.7 

1.44 

3.1 

1.55 

3.26 

0.075 

information.  The  mean  score  for  the  high- impact  cases  (1.95) 
lies  just  to  the  left  of  the  somewhat  easy  rating,  while  the 
low-impact  scores  (2.46)  falls  between  the  somewhat  easy  and 
neutral  rating.  The  results  show  that  the  high- impact  style 
makes  it  easier  to  apply  the  adjudication  criteria. 

(3)  Questions  5.  6.  and  7:  Perception  of  Field 

Agents  Awareness 

Questions  5,  6  and  7  gauge  the  adjudicators' 
perception  of  field  agents'  awareness  of  their  needs. 
Specifically,  the  questions  asked  the  adjudicators,  based  on 
the  way  information  was  presented  in  the  cases,  did  they  feel 
the  field  agent  seemed  to  be  aware  or  unaware  of  the 
following: 

•  Question  5:  The  information  needed  to  make  a  high  quality 
adjudication  decision; 

•  Question  6 :  How  to  make  the  case  easier  to  read; 

•  Question  7 :  The  large  amounts  of  daily  reading  done  by  the 
adjudicators. 

The  high- impact  cases  mean  scores  are  1.73  for  question  5, 
2.05  for  question  6,  and  2.03  for  question  7.  These  scores 
all  fall  within  the  area  of  the  "somewhat  aware"  rating 
category.  In  contrast,  the  low-impact  cases  mean  scores  are 
2.18  for  question  5,  2.58  for  question  6,  and  2.63  for 

question  7.  These  results  lie  between  the  somewhat  aware  and 
neutral  rating  categories. 
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Although  the  adjudicators  perception  of  the 
field  agents  who  wrote  the  cases  are  tested  much  more 
thoroughly  in  Questionnaire  Part  III,  the  results  of  these 
three  questions  point  out  that  document  design,  organization 
and  writing  style  do  affect  readers'  perception  of  the 
writer's  awareness.  The  author  of  a  high- impact  style 
document  is  perceived  as  having  a  greater  awareness  of  the 
reader's  needs  than  the  author  of  a  low-impact  one. 

(A)  Question  9 :  Evaluation  of  Narrative  Report 

Wri ters 


Question  8  asked  adjudicators  if  they  believed 
that  the  field  agents  who  wrote  the  narrative  reports  for  the 
cases  were  good  or  poor  writers.  The  high- impact  mean  score 
(1.93)  falls  just  to  the  left  of  the  somewhat  good  rating, 
while  the  low-impact  score  (2.5)  lies  exactly  in  the  middle  of 
the  somewhat  good  to  neutral  rating.  The  result  to  this 
question  showed  the  greatest  statistical  significance.  They 
are  significant  at  the  .6%  confidence  level.  These  results 
show  that  the  majority  of  adjudicators  believe  that  the 
authors  of  the  high  intact  narrative  reports  are  better 
writers  than  the  authors  of  low  impact  reports. 

(5)  Question  11.  and  12:  Ease  of  Finding  and 
Reading  Derogatory  Information 

Questions  11  and  12  tested  the  ease  of  finding 
and  reading  derogatory  information  in  the  cases.  The  high- 


36 


impact  mean  scores  are  1.9  for  question  11  and  1.83  for 
question  12,  which  both  fall  to  the  left  of  the  somewhat  easy 
rating.  The  low-impact  mean  scores  are  2.5  for  question  11 
and  2.33  for  question  12,  which  both  fall  between  the  somewhat 
easy  to  neutral  rating. 

These  results  indicate  that  adjudicators  found 
the  high- impact  style  cases  made  finding  and  reading 
information  easier.  Some  of  the  specific  characteristic  of  a 
high- impact  document  that  make  finding  and  reading  information 
easier  include  the  use  of 

•  heading  and  subheadings 

•  lists  and  bullets 

•  internal  previews 

•  active  verbs 

•  and  explicit  transitional  tags. 

The  results  show  that  these  and  other  high- 
impact  stylistic  strategies  created  the  perception  that 
finding  and  reading  derogatory  information  was  easier. 

(6)  Questions  14.  Id.  and  19:  Ease  of  Remembering, 
Locating  and  Following  Presented  Case  Information 

These  questions  asked  the  adjudicators  the 
relative  ease  they  had  in  the  following  areas: 

•  Question  14;  remembering  the  presented  case  information, 

•  Question  16;  locating  the  essential  information  necessary 
for  the  efficient  adjudication  of  the  case. 
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•  Question  19:  following  the  chronological  sequence  of 
events  in  the  case. 

The  high- impact  cases  mean  scores  are  1.9  for  question  14,  1.8 
for  question  16,  and  2.05  for  question  19.  These  scores  all 
fall  in  the  area  of  the  somewhat  easy  rating  category.  The 
low-impact  cases  mean  scores  are  2.5  for  question  14,  2.48  for 
question  16,  and  2.6  for  question  19.  These  results  cluster 
between  the  somewhat  easy  and  neutral  rating  categories. 

The  results  show  that  adjudicators  feel  that 
information  presented  in  a  high- impact  style  is  easier  to 
remember  and  locate  than  information  presented  in  low  impact 
style.  They  also  indicate  that  the  high- impact  style  makes  it 
easier  to  follow  chronological  sequences  of  events.  This  can 
be  interrupted  to  mean  that  the  high  intact  style  increases 
information  retention  in  short  term  memory,  improves 
efficiency  in  locating  essential  information  in  the  document, 
as  well  as  improve  their  cdjility  to  follow  a  sequence  of 
events . 

(7)  Question  15:  Ease  of  Adjudicating  Case 

The  adjudicators  were  asked  in  this  question 
whether  they  found  it  easy  or  difficult  to  adjudicate  the 
case.  The  high- inpact  mean  score  (1.78)  falls  just  to  the 
left  of  the  somewhat  easy  rating,  while  the  low-impact  score 
(2.3)  lies  between  the  somewhat  easy  to  neutral  rating. 

The  result  cuiplifies  the  finding  to  questions 
14,  16,  and  19.  Ease  in  locating  and  remembering  essential 
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case  information,  and  following  chronological  events  should 
leads  to  the  easier  adjudication  of  cases. 

(8)  Question  22:  Fatigue  of  Reading 

Question  22  asked  the  adjudicators  whether  they 
agreed  or  disagreed  with  the  statement  "Reading  this  case 
tired  me  out."  The  high-intact  mean  score  (4.38)  falls 
between  the  somewhat  disagree  and  disagree  rating  categories, 
while  the  low-impact  score  (3.78)  lies  between  the  neutral 
rating  and  the  somewhat  disagree  ratings. 

The  above  results  indicate  that  the  high- impact 
cases  caused  less  fatigue  than  the  low- intact  ones.  The  high- 
intact  document  factors  that  may  contribute  to  this  decreased 
fatigue,  include  the  use  of  headings  and  siibheadings ,  as  well 
as  bullets  and  lists. 

Jb.  Questions  Statistically  Significant  within  the  10% 
Confidence  Interval 

Although  the  results  to  c[uestions  3,  4,  18,  and  23, 
did  not  meet  the  5%  criteria,  they  did  show  significance 
within  the  10%  confidence  interval.  Questions  3  and  4  tested 
the  adjudicators  confidence  in  the  quality  of  the  field 
agents'  investigation  and  the  soundness  of  their  adjudication 
decision.  The  frequency  of  rereading  sentences  was  the 
objective  of  question  18,  while  eye  strain  was  the  factor 
tested  for  in  question  23. 
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The  high- impact  cases  mean  scores  are  1.93  for 
question  3,  1.38  for  question  4,  3.88  for  question  18,  and  3.7 
for  question  23.  The  high- impact  scores  for  both  questions  3 
and  4,  fall  between  confident  and  somewhat  confident  rating 
category,  question  18  scores  lie  between  the  somewhat  never  to 
never  rating,  and  question  23  scores  fall  between  the  somewhat 
disagree  to  disagree  ratings. 

The  low-impact  cases  mean  scores  are  2.35  for 
question  3,  1.65  for  question  4,  3.45  for  question  18,  and  3.1 
for  question  23.  The  low- intact  scores  for  questions  3  fall 
between  the  somewhat  confident  and  neutral  rating  category, 
question  4  falls  between  the  confident  and  somewhat  confident 
ratings,  question  18  scores  lie  between  the  somewhat  never  to 
never  rating,  and  question  23  scores  fall  between  the  somewhat 
disagree  to  disagree  ratings. 

In  all  of  the  above  questions,  it  is  inferred  that 
the  design,  organization,  writing  style  of  the  high- impact 
cases  increased  1)  the  confidence  in  the  field  agent's 
investigation  and  adjudicators'  decision,  2)  reduced  the 
frequency  of  sentence  rereading,  and  3)  reduced  eye  strain. 

c.  Sumaetry  of  Both  the  Czaxnek  and  Rokitka  Case 

Results 

When  we  combine  the  results  of  the  thirteen 
questions  that  passed  the  5%  confidence  test,  and  the  four 
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that  passed  the  10%  test,  it  is  clear  that  adjudicators  feel 
that  the  document  design,  organization,  and  writing  style  of 
the  high- impact  cases  to  be  superior  to  the  currently  used 
low-impact  style  cases.  As  will  be  seen  in  the  following 
sections,  this  result  is  not  necessarily  true  when  the  Czarnek 
and  Rokitka  cases  or  the  adjudications  sites  are  analyzed 
separately. 

3.  Analysis  of  the  High  and  Low-Iaqpact  Czamek  Cases 

The  results  of  the  analysis  of  the  high  and  low-impact 
Czarnek  cases  are  presented  in  Table  4-2.  Out  of  the  twenty- 
three  questions  analyzed,  none  of  the  results  fell  within  the 
5%  confidence  interval  and  only  four  (#'s  2,  8,  16,  and  22) 
met  the  10%  confidence  criteria.  The  following  is  a  summary 
of  each  of  the  four  questions: 

•  Question  2 :  How  easy  was  it  to  applying  the  adjudication 
criteria  to  the  case  information? 

•  Question  8:  What  was  the  quality  (good  to  poor)  of  the 
narrative  report  writers? 

•  Question  16:  How  easy  was  it  to  locate  essential  case 
information? 

•  Question  22:  Was  reading  the  case  tiring? 

Comparing  the  result  to  these  questions  with  those 
from  the  analysis  of  both  cases,  we  see  consistent  findings. 
That  means  that  the  aJaove  statistically  significant  questions 
were  also  found  significant  within  either  the  5%  or  10% 
interval  in  the  combined  Czarnek  and  Rokitka  analysis  (Table 
4-1) 
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TABLE  4-2 


SUMMARY  OF  QUESTIONNAIRE 
ANALYSIS  OF  THE  CZARNEK 
ADJUDICATOR'S  IMPRESSION 

PART  II 
CASE  (n 
rs  OF  CAS 

RESULTS 
=  20) 

E  INFORM? 

lTION 

QUEST 

# 

HI 

MEAN 

HI  S.D. 

LI 

MEAN 

LI  S.D. 

F-STAT 

SIG. 
of  F 

1 

2.21 

1.08 

2.65 

0.88 

1.59 

0.216 

2 

2.16 

1.17 

2.74 

0.93 

3.21 

0.082 

3 

2.15 

1.34 

2.6 

0.88 

1.89 

0.178 

4 

1.5 

1 

1.8 

1.01 

1.46 

0.235 

5 

1.95 

1.05 

2.3 

1.08 

1.09 

0.304 

6 

2.16 

1.17 

2.5 

0.76 

1.19 

0.283 

7 

2.2 

1.32 

2.65 

1.09 

2.71 

0.108 

8 

1.95 

1.05 

2.45 

0.69 

3.87 

0.057 

9 

3.65 

1.42 

3.55 

0.89 

0.12 

0.728 

10 

2.75 

1.25 

3.05 

0.76 

1.78 

0.19 

11 

2.05 

1.32 

2.45 

1 

1.17 

0.286 

12 

1.95 

1.15 

2.35 

0.99 

1.4 

0.244 

13 

2.15 

1.39 

2.8 

1.2 

2.52 

0.121 

14 

2.1 

1.29 

2.45 

1 

0.92 

0.344 

15 

1.9 

1.17 

2.4 

1.19 

1.81 

0.187 

16 

1.8 

1.15 

2.45 

1.05 

3.48 

0.07 

17 

3.3 

1.08 

3.35 

0.99 

0.023 

0.879 

18 

3.85 

0.99 

3.6 

0.82 

0.758 

0.39 

19 

2.25 

1.16 

2.55 

1.32 

0.583 

0.45 

20 

3.6 

1.04 

3.15 

1.09 

1.78 

0.191 

21 

4.7 

0.57 

4.7 

0.57 

0 

1 

22 

4.55 

0.76 

3.9 

1.25 

3.94 

0.054 

23 

3.4 

1.54 

1.6 

1.23 

0.274 
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Given  that  there  were  only  four  statistically 
significant  questions  overall,  and  all  of  these  fell  only 
within  the  10%  interval,  these  results  tell  us  that  there  was 
not  a  strong  difference  in  adjudicators'  impressions  of  the 
high  and  low-impact  Czarnek  cases.  It  can  be  further  inferred 
that  Czarnek  results  are  not  the  driving  force  behind  the 
large  amount  of  statistical  significance  found  when  both  the 
Czarnek  and  Rokitka  Cases  were  analyzed  together.  This  is 
counter-intuitive  to  expected  results,  because  the  Czarnek 
case  content  is  more  complicated  and  longer  than  Rokitka  case 
content.  Common  sense  would  dictate  that  the  longer  and  more 
complicated  a  document's  content  is,  the  more  helpful  the 
high- impact  style  would  be  in  reading,  finding  and  locating 
information. 

4.  Analysis  o£  the  High  and  LowIaq>act  Rokitka  Cases 

The  results  of  the  high  and  low-inpact  Rokitka  case 
data  are  presented  in  Table  4-3.  Questions  8,  12,  14,  and 
19were  found  to  be  statistically  significant  within  the  5% 
interval  while  questions  1,  5,  6,  7,  11,  15,  and  16  were 

significant  within  the  10%  interval.  The  following  is  a 
svimmary  of  each  of  the  four  questions  that  met  the  5% 
confidence  criteria; 

•  Question  8 ;  What  was  the  quality  (good  to  poor)  of  the 
narrative  report  writers? 

•  Question  12 ;  How  easy  was  it  to  read  the  derogatory  case 
information? 
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TABLE  4-3 


SUMMARY 

ANALYS] 

ADJUDIC 

r  OF  QUESTIONNAIRE  PART  I I 
:S  OF  THE  ROKITKA  CASE  (n 
[TUTOR'S  IMPRESSIONS  OF  CAS 

RESULTS 
=  20) 

E  INFORM? 

O 

H 

QUEST 

# 

HI 

MEAN 

HI  S.D. 

LI 

MEAN 

LI  S.D. 

F-STAT 

SIG. 
of  F 

1 

1.6 

0.88 

2.15 

1.09 

3.08 

0.087 

2 

1.75 

1.12 

2.2 

1.01 

1.79 

0.189 

3 

1.7 

0.98 

2.1 

0.85 

1.9 

0.176 

4 

1.25 

0.44 

1.5 

0.69 

1.86 

0.18 

5 

1.5 

0.76 

2.05 

1.05 

3.6 

0.065 

6 

1.95 

1.19 

2.65 

1.27 

3.24 

0.08 

7 

1.85 

1.09 

2.6 

1.27 

4.01 

0.053 

0 

iiiiiiiii 

0.9 

2.55 

1.1 

4,14 

0,049 

9 

3.15 

1.66 

3.45 

1.23 

0.42 

0.521 

10 

2.35 

1.31 

2.65 

1.18 

0.58 

0.452 

11 

1.75 

0.79 

2.55 

1.23 

5.98 

0.091 

12 

1.7 

0.73 

2.3 

0.92 

5,18 

0,029 

13 

2.3 

1.42 

2.34 

1.14 

0.015 

0.903 

"i4'': 

2.55 

1,4 

5,37 

0,024 

15 

1.65 

0.88 

2.2 

1.15 

2.89 

0.097 

16 

1.8 

0.89 

2.5 

1.28 

4.03 

0.052 

17 

3.8 

1.11 

3.2 

1.28 

2.52 

0.121 

18 

3.9 

1.17 

3.3 

1.34 

2.28 

0.139 

1^ 

l«04f:  ■ 

2. $5 

1.31 

4.58 

Bill 

20 

3.45 

1.05 

3.2 

1.15 

0.515 

0.478 

21 

4.65 

0.74 

4.25 

0.91 

2.31 

0.137 

22 

mem 

1.11 

3.65 

1.27 

2.14 

0.152 

23 

4 

1.3 

3.35 

1.5 

2.15 

0.15 
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•  Question  14:  How  easy  was  it  to  remember  case  information? 


•  Question  19:  How  easy  was  it  to  follow  chronological 
events  in  the  case? 

Questions  meeting  the  10%  criteria  asked  the  following: 
question  1,  the  coverage  of  issues  in  the  case;  question  5  and 
7,  the  field  agents'  awareness  of  the  adjudicators' 
information  needs  and  daily  reading  requirements;  question  6, 
the  field  agents  ability  of  make  the  case  easier  to  read;  and 
finally,  Questions  11,  15  and  16:  the  ease  in  locating 
derogatory  and  essential  information  as  well  as  adjudicating 
the  case. 

Comparing  the  result  to  these  questions  with  those 
from  the  analysis  of  both  cases,  we  see  consistent  findings. 
In  other  words,  the  statistically  significant  response  to 
questions  listed  above  were  also  found  significant  within 
either  the  5%  or  10%  confidence  interval  in  the  combined 
Czarnek  and  Rokitka  analysis  (Table  4-1) . 

Comparing  Rokitka  case  results  to  the  Czarnek  (Teible 
4-3)  shows  that  the  document  design,  organization,  and  writing 
style  of  the  Rokitka  cases  had  a  greater  statistical 
significance  on  the  adjudicator's  impressions  of  both  the  high 
and  low-impact  writing  style  than  the  Czarnek  Cases.  This 
comparison  makes  clear  that  the  Rokitka  cases  are  the  driving 
force  behind  the  statistically  significant  results  of  the 
analysis  of  both  cases. 
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5.  Analysis  o£  the  Separate  Adjudication  Sites  using  both 
the  High  and  Low- intact  Czamek  euid  Rokltka  Cases 

This  section  presents  and  analyzes  the  separate 
adjudication  site  results.  There  are  three  subsections  in 
this  section;  each  subsection  presents  and  analyzes  the  result 
for  the  Army,  Navy  or  DISCR  sites  respectively. 

a.  Presentation  and  Analysis  of  the  Army  Adjudication 
Site  Results 

Results  from  the  Army  Adjudication  Site  produced 
more  statistically  significant  results  than  the  Navy  and  DISCR 
Sites  combined.  Table  4-4  shows  that  the  results  to  three 
questions  (#'s  1,  4,  and  12)  were  significant  within  the  5% 
confidence  interval,  while  five  (#'s  6,  8,  10,  14,  and  20) 
were  within  10%.  The  following  is  a  siunmary  of  each  of  the 
three  questions  that  met  the  5%  confidence  interval  criteria. 

•  Question  1:  What  was  the  quality  (good  to  poor)  of  the 
coverage  of  issues  in  the  case? 

•  Question  4:  How  confident  was  your  adjudication  decision? 

•  Question  12:  How  easy  was  it  to  read  the  derogatory  case 
information? 

Questions  meeting  the  10%  criteria  asked  about  the  following: 
question  6,  the  field  agents'  ability  of  make  the  case  easier 
to  read;  question  8,  the  quality  of  narrative  report  writer; 
question  10,  the  perceived  speed  of  reading  the  case; 
questions  14,  the  ease  of  remembering  derogatory  information 
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TABLE  4-4 


SUMMARY  OF  QUESTIONNAIRE 
ANALYSIS  OF  ARMY  SITE  (n 
ADJUDICATOR'S  IMPRESSION 

PART  II  RESULTS 

L  =  12)  USING  BOTH  CASES 
rS  OF  CASE  INFORMATION 

QUEST 

# 

HI 

MEAN 

HI  S.D. 

LI 

MEAN 

LI  S.D. 

F - STAT 

SIG. 

Of  F 

X 

1.82 

0.75 

a. 58 

0,8 

4,61 

0.045 

2 

2.08 

1.08 

2.58 

1 

1.58 

0.224 

3 

2.08 

1.08 

2.67 

0.89 

2.33 

0.144 

4 

1.33 

0,65 

1,82 

1 

0,044 

5 

1.83 

0.83 

2.25 

1.14 

1.47 

0.24 

6 

1.9 

0.94 

2.58 

0.9 

2.98 

0.1 

7 

2.08 

1.31 

2.5 

0.9 

2.7 

0.117 

8 

1.92 

0.79 

2.42 

0.67 

3.79 

0.066 

9 

3.42 

1.44 

3.5 

0.8 

0.006 

0.94 

10 

2.5 

1.24 

2.92 

0.67 

3.05 

0.097 

11 

2.17 

1.11 

2.67 

0.78 

1.58 

0.223 

12 

1  1.S3 

0,83 

2.58 

1  4.77 

0,041 

13 

2.17 

1.11 

2.67 

1.07 

1.2 

0.286 

14 

1.92 

0.9 

2.67 

1.23 

3.29 

0.085 

15 

1.83 

0.94 

2.33 

1.07 

1.55 

0.227 

16 

2 

1.04 

2.5 

0.9 

1.48 

0.239 

17 

3.42 

1 

3.08 

1.38 

0.43 

0.517 

18 

3.75 

0.97 

3.42 

1.24 

0.5 

0.485 

19 

1.92 

0.9 

2.5 

1.31 

1.6 

0.22 

20 

3.67 

0.78 

3 

0.85 

3.72 

0.068 

21 

4.67 

0.65 

4.33 

0.78 

1.38 

0.254 

22 

4.33 

0.78 

3.83 

1.11 

1.7 

0.207 

23 

3.83 

1.47 

3.25 

1.42 

0.953 

0.341 
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in  the  case;  and  finally,  question  20,  the  frequency  of 
underlining  or  note  taking. 

Comparing  the  above  results  with  those  from  the 
analysis  of  both  cases  using  the  entire  adjudicator 
population,  we  see  three  key  differences.  The  first  two  are 
that  the  results  of  both  question  10  and  20  were  found  to  be 
statistically  insignificant  when  both  cases  were  analyzed. 
The  third  difference  is  that  the  result  to  Question  4  fell 
within  the  5%  confidence  interval  for  this  site  but  was  only 
at  the  10%  level  in  Table  4-1. 

All  other  statistically  significant  results  from 
the  Army  sites  are  consistent  with  the  combine  case  analysis. 
That  means  that  excluding  the  above  exceptions,  all  questions 
found  significant  within  either  the  5%  or  10%  confidence 
interval  were  also  significant  in  the  combined  Czarnek  and 
Rokitka  analysis  (Table  4-1) . 

We  also  found  that  there  were  several  question  that 
fell  within  the  insignificant  range  in  the  Army  analysis  but 
significant  in  the  analysis  of  both  cases.  These  results  may 
be  caused  by  the  smaller  sample  size  of  the  Army  Adjudicator 
test  population. 

We  are  uncertain  why  the  Army  site  has  both  an 
abundance  of  significant  results,  and  notable  statistical 
inconsistences  with  the  results  of  the  combined  Czamek  and 
Rokitka  cases. 
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Jb.  Presentation  and  Analysis  of  the  Navy  Adjudication 

Site  Results 

The  results  of  the  Navy  site  analysis  are  presented 
in  Table  4-5.  Only  the  result  to  question  #2  was 
statistically  significant  within  the  5%  confidence  interval, 
and  four  (#'s  1,  7,  8,  and  16)  were  within  the  10%  interval. 
Question  2  asked  adjudicators  if  they  felt  it  was  easy  or 
difficult  to  apply  the  adjudication  criteria.  The  questions 
meeting  the  10%  criteria  asked  the  following:  question  1,  the 
coverage  of  issues  in  the  case;  question  7,  the  field  agents 
awareness  of  the  adjudicators'  daily  reading  requirement; 
question  8,  the  quality  of  narrative  report  writer;  and 
finally.  Questions  16,  the  ease  in  locating  essential  case 
information. 

Comparing  the  above  results  with  those  from  the 
analysis  of  both  cases  using  the  entire  adjudicator  population 
we  find  the  results  are  mostly  consistent.  The  only 
exceptions  are  that  there  are  fewer  statistically  significant 
result  in  the  Navy  Site  analysis,  and  questions  1,7,  8  and  16 
fell  within  the  10%  criteria  at  the  Navy  Site  but,  were  at  the 
5%  level  when  treatment  of  both  cases  were  combined. 

c.  Presentation  and  Analysis  of  the  DISCR 

Adjudication  Site  Results 

The  result  of  the  DISCR  site  analysis  are  presented 
in  Table  4-6.  This  site  produced  the  least  amount  of 
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TABLE  4-5 


SUMMARY 

ANALYS] 

ADJUDIC 

r  OF  QUESTIONNAIRE 
CS  OF  NAVY  SITE  (n 
V^TOR'S  IMPRESSION 

PART  II  RESULTS 

L  =  12)  USING  BOTE 
fS  OF  CASE  INFORMS 

CASES 

iTION 

QUEST 

# 

HI 

MEAN 

HI  S.D. 

LI 

MEAN 

LI  S.D. 

F-STAT 

SIG. 
of  F 

1 

1.92 

0.79 

2.67 

1.23 

4.14 

0.056 

a 

1,75 

2.^4 

0.92 

6.11 

0,023 

3 

1.92 

0.9 

2.42 

0.9 

2.85 

0.108 

4 

1.33 

0.49 

1.33 

0.49 

0.013 

0.912 

5 

1.83 

0.72 

2.42 

1.08 

2.67 

0.119 

6 

2.25 

1.05 

2.67 

0.89 

1.1 

0.305 

7 

2.33 

1.07 

3.25 

1.36 

3.1 

0.093 

8 

2 

0.85 

2.67 

0.89 

3.27 

0.086 

9 

3.42 

1.56 

3.5 

1 

0.023 

0.882 

10 

2.58 

1.16 

2.92 

1 

0.548 

0.468 

11 

2 

1.04 

2.5 

1.31 

0.989 

0.332 

12 

1.83 

0.83 

2.17 

1.03 

0.708 

0.412 

13 

2.33 

1.5 

2.42 

0.79 

0.027 

0.872 

14 

1.83 

0.94 

2.58 

1.24 

2.55 

0.126 

15 

1.67 

0.78 

2.42 

1.31 

2.68 

0.117 

16 

1.75 

0.62 

2.58 

1.38 

3.33 

0.083 

17 

3.67 

1.07 

3.08 

0.89 

0.635 

0.435 

18 

3.92 

1.08 

3.42 

0.9 

0.954 

0.34 

19 

1.83 

1.11 

3.25 

1.44 

1.96 

0.177 

20 

2.58 

1.05 

3.33 

1.15 

0.033 

0.858 

21 

4.42 

0.9 

4.58 

0.67 

0.256 

0.618 

22 

4.25 

0.96 

4.08 

1.08 

0.159 

0.695 

23 

^3^5 

^1^31 

3^3^ 

1.56 

^^07^ 

0.782 

TABLE  4-6 


SUMMARY  OF  QUES 
ANALYSIS  OF  DIS 
ADJUDICATOR'S  I 

TIONNAIRE 
CR  SITE  ( 
MPRESSIO« 

PART  II 
n  =  12) 
rS  OF  CAS 

RESULTS 
USING  BOI 
E  INFORMS 

H  CASES 
lTION 

QUEST 

HI 

MEAN 

HI  S.D. 

LI 

MEAN 

LI  S.D. 

F-STAT 

SIG. 
of  F 

1 

1.75 

1.36 

2.08 

0.9 

0.2-i 

0.63 

2 

2 

1.55 

2.33 

1.15 

0.282 

0.602 

3 

1.67 

1.23 

12.17 

0.94 

0.78 

0.389 

4 

1.33 

1.15 

1.67 

0.98 

0.358 

0.556 

5 

1.5 

1.24 

1.92 

1.16 

0.438 

0.516 

6 

2 

1.48 

2.5 

1.38 

0.422 

0.422 

7 

1.75 

1.36 

2.25 

1.22 

0.826 

0.374 

8 

1.75 

1.22 

2.5 

1.24 

2.056 

0.167 

9 

3.5 

1.73 

3.92 

1.24 

0.475 

0.498 

10 

2.33 

1.44 

2.67 

1.37 

0.31 

0.584 

11 

1.58 

1.16 

2.42 

1.38 

2.34 

0.142 

12 

1.75 

1.22 

2.25 

1.14 

0.989 

0.332 

13 

2.17 

1.64 

2.58 

1.51 

0.41 

0.529 

14 

1.83 

1.4 

2.42 

1.31 

1.02 

0.325 

15 

1.67 

1.37 

2.25 

1.29 

1.05 

0.317 

16 

1.67 

1.23 

2.42 

1.38 

1.8 

0.195 

17 

3.75 

1.22 

3.42 

1.31 

0.408 

0.532 

18 

2.25 

0.97 

3.58 

1.31 

1.88 

0.185 

19 

2.33 

1.37 

2.67 

1.37 

0.328 

0.573 

20 

3.5 

1.31 

3.17 

1.4 

0.357 

0.557 

21 

4.92 

0.29 

4.67 

0.89 

0.918 

0.349 

0.028 

23 

4 

1.28 

2.83 

3.14 
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statistically  significant  differences  (i.e.  two  questions) . 
The  results  to  one  question  (#22)  was  statistically 
significant  within  the  5%  confidence  interval  while  the  other 
questions  (#23)  fell  within  the  10%  interval.  Questions  22 
and  23  asked  the  adjudicators  if  reading  the  case  caused 
fatigue  and  eye  strain  respectively.  Comparing  the  above 
results  with  those  from  the  analysis  of  both  cases  using  the 
entire  adjudicator  population,  we  see  that  the  results  are 
consistent. 

We  must  note  that  the  results  to  the  other  twenty- 
one  questions  were  well  outside  of  the  10%  interval.  From 
this,  we  can  infer  that  there  is  relatively  little  difference 
in  the  DISCR  adjudicators'  iit5)ression  of  the  case  information 
between  the  two  styles.  This  lack  of  difference  is  far 
greater  than  results  from  either  the  Army  or  Navy  Adjudication 
Sites. 

6.  Presentation  and  Analysis  of  the  Male  Adjudicator 
Results 

As  described  in  the  third  chapter,  an  analysis  of 
several  demographic  breakdowns  of  the  adjudicator  population 
was  performed.  Similar  to  the  results  found  in  the  analysis 
of  the  adjudicating  times,  the  only  group  to  produce  a  large 
amount  of  statistically  significeuatly  results  was  the  male 
adjudicators . 
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A  presentation  of  these  results  are  in  Table  4-7.  The 
results  to  nine  questions  (#'s  1,  2,  3,  6,  7,  11,  14,  15,  16, 
and  18)  were  found  to  be  statistically  significant  within  5%, 
and  five  question  (#'s  5,  8,  12,  17,  and  19)  within  10%.  The 
following  is  a  summary  of  the  nine  question  that  met  the  5% 
confidence  interval  criteria: 

•  Question  1:  What  was  the  quality  (good  to  poor)  of  the 
coverage  of  issues  in  the  case? 

•  Question  2 :  How  easy  was  it  to  apply  the  adjudication 
criteria  to  the  case  information? 

•  Question  3 :  How  confident  were  you  in  the  quality  of  the 
field  agent's  investigation? 

•  Question  6:  How  aware  were  field  agents  in  making  the  case 
easier  to  read? 

•  Question  7 :  How  aware  was  the  field  agent  to  the 
adjudicator  daily  reading  requirement? 

•  Question  lit  How  easy  was  it  to  find  derogatory 
information? 

•  Question  14 1  How  easy  was  it  to  remember  derogatory 
information? 

•  Question  15:  How  easy  was  it  to  adjudicate  the  case? 

•  Question  16 :  How  easy  was  it  to  locate  essential  case 
information? 

•  Question  18:  How  often  were  sentences  reread? 

The  questions  meeting  the  10%  criteria  asked  the  following: 
question  5,  the  field  agents  awareness  of  the  adjudicators 
information  needs;  question  8,  the  quality  of  narrative  report 
writer;  questions  12,  the  ease  of  reading  derogatory 
information  in  the  case;  question  17,  the  frequency  of  reading 
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TABLE  4-7 


SUMMARY  OF  QUESTIONNAIRE 
ANALYSIS  OF  MALE  ADJUDIC 
ADJUDICATOR'S  IMPRESSION 

PART  II 
ATORS  (n 
rs  OF  CAS 

RESULTS 
=  9)  USI 
E  INFORM? 

•NG  BOTH  C 
lTION 

ASES 

QUEST 

# 

HI 

MEAN 

HI  S.D. 

LI 

MEAN 

LI  S.D. 

F-STAT 

SIG. 
of  F 

a. 

1«  44 

a*55 

0.017 

2 

0*$3 

3 

0*004 

1*32 

0,6 

2.55 

1.01 

9*19 

4 

1.22 

0.44 

1.67 

1 

1.41 

0.254 

5 

1.44 

0.73 

2.67 

1.41 

4.5 

0.052 

1.66 

o*s 

3 

11.59 

0*004 

7 

0.93 

3.22 

0.027 

8 

1.89 

0.78 

2.78 

0.97 

4.46 

0.053 

9 

3.89 

1.17 

3.67 

1.22 

0.347 

0.565 

10 

2.22 

0.83 

3 

1.22 

2.32 

0.15 

Hlil 

1*64 

0.5 

2.89 

1,17 

7*72 

12 

1.67 

0.5 

2.56 

1.24 

4.12 

0.06 

13 

2.11 

1.36 

3 

1.22 

2.45 

0.14 

14 

1.78 

0*83 

3 

1*'22 

5.57 

IS 

0.73 

2.89 

1.27 

7*87 

.  0.018 

lilies 

•  1.67' 

o*s 

1.12 

9.41 

0*008 

17 

3.78 

1.09 

2.89 

1.17 

4.18 

■xa 

3.69 

0*78 

2*89 

1*17 

7.42 

0*015 

19 

1.78 

0.44 

2.78 

1.3 

4.01 

0.065 

20 

3.22 

1.39 

3 

1.22 

0.18 

0.674 

21 

4.89 

0.33 

4.78 

0.44 

0.41 

0.53 

22 

4.33 

0.5 

4 

1.32 

0.71 

0.413 

4 

3.11 

1.69 

1.78 

0.203 
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section  in  the  case;  and  finally,  question  19,  the  ease  of 
following  the  chronological  sequence  of  events  in  the  case. 
Comparing  the  above  results  with  those  from  the 
analysis  of  both  cases  using  the  entire  adjudicator 
population,  we  once  again  see  that  the  results  are  consistent. 
These  results,  however,  are  quite  surprising  considering  the 
small  sample  size  (n=9)  of  male  adjudicators. 

C.  CHAPTER  CONCLUSION 

The  results  from  the  high  and  low-impact  treatment  of  both 
the  Czarnek  and  Rokitka  cases  point  out  that  there  is  no 
statistical  evidence  to  show  that  the  high- impact  decreases 
adjudication  time.  Conversely,  there  is  evidence  that 
adjudicators  feel  that  the  high- impact  style  is  an  iirprovement 
over  the  low-impact  one.  Specifically,  they  believe  that  the 
high- impact  style 

•  provides  better  coverage  of  issues; 

•  makes  it  easier  to  apply  adjudication  criteria  to  the  case 
information; 

•  makes  the  field  agent  who  wrote  the  case  seem  more  aware 
of  1)  how  to  make  the  case  easier  to  read;  2)  their 
informational  needs;  3)  the  extensive  amount  of  their 
daily  reading; 

•  makes  the  field  agent  seem  like  a  better  writer; 

•  makes  the  derogatory  information  in  the  narrative  report 
easier  to  find,  read,  remember  and  adjudicate; 

•  makes  following  the  chronological  sequence  of  events  in 
the  case  easier  to  follow; 

•  and  causes  less  fatigue  while  reading. 
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To  a  lesser  degree  adjudicators  believe  that  the  high- 
impact  style  makes  them  more  confident  in  the  soundness  and 
quality  of  their  adjudication  decision,  causes  them  to  reread 
sentences  less  often,  and  produces  less  eye  stain. 

We  also  found  that  there  were  greater  statistically 
significant  differences  in  the  adjudicators  impressions  of  the 
case  information  in  the  Rokitka  cases  than  the  Czamek  cases. 
This  is  contrary  to  expected  results  because  the  Czamek  case 
content  is  more  complicated  and  longer  than  Rokitka  case 
content.  One  would  expect  chat  the  longer  and  more 
complicated  a  documents  content  is  the  more  helpful  the  high- 
intact  style  would  be  in  reading,  finding,  and  locating 
information. 

When  the  analysis  was  divided  by  adjudication  site,  the 
statistical  significance  was  not  nearly  as  great  as  the  result 
from  the  entire  population.  However,  this  could  be  attributed 
to  the  smaller  sample  sizes  used.  The  greatest  statistical 
differences  were  found  from  the  analysis  of  the  Army 
adjudication  site  data.  The  Navy  adjudication  site  data 
produced  the  second  most  statistically  significant  result, 
while  DISCR  data  had  the  least. 

The  most  surprising  result  was  found  when  analyzing  the 
demographic  breakdown  of  adjudicators.  The  male  adjudicators 
found  that  the  high- impact  style  decreased  the  time  required 
to  adjudicate  a  case.  They  also  felt  that  the  high- impact 
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cases  were  a  much  greater  improvement  over  the  current  low- 
impact  cases  than  their  female  counterpart. 
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V.  PRESENTATION  AND  ANALYSIS  OF  PART  III  RESULTS:  THE 
ADJUDICATOR'S  PERCEPTION  OF  THE  FIELD  AGENT 

Questionnaire  Part  III  (see  Appendix  B)  tests 
adjudicators'  perception  of  or  feeling  towards  the  field 
agents  who  wrote  either  the  high- impact  or  low- intact  ROI. 
This  survey  section  asked  adjudicators  to  rate  the  field 
agents  using  fifteen  bipolar  adjectives  such  as  objectivity, 
clarity  of  thinking,  reliability,  etc..  A  five-point  bipolar 
scale  was  used  to  capture  the  adjudicator's  feelings. 
Specifically,  this  part  of  the  survey  was  designed  to 
determine  if  the  high  or  low-impact  style  caused  differences 
in  adjudicators'  perceptions  of  field  agents. 

This  chapter  presents  and  analyzes  the  results  to 
Questionnaire  Part  III  in  the  same  manner  as  the  Part  II 
results  in  the  previous  chapter.  The  first  section  of  the 
chapter  gives  a  very  brief  explanation  of  how  to  read  the 
tables  of  results,  cuid  the  last  draws  conclusion.  The 
remaining  middle  sections  cover  the  analysis  results  from  both 
cases,  the  Czamek  cases,  the  Rokitka  cases,  the  separate 
adjudication  sites,  and  finally  the  male  adjudicators. 

A.  DIRECTIONS  FOR  READING  TABLES  OF  RESULTS 

Similar  to  the  previous  chapter,  there  are  seven  teJsles  of 
results  in  this  chapter.  These  tables  will  follow  the  same 
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format  as  the  previous  tables  (i.e.  title  block,  column 
headings,  shading  of  5%  results,  and  bolding  of  10%  results)  . 
The  only  differences  are  the  information  in  the  title  block, 
the  results  to  the  specific  questions,  and  total  number  of 
questions  (15  vice  23) . 

B.  ANALYSIS  OF  BOTH  CZABNEK  AND  ROKITKA  CASES 

The  thesis's  second  primary  research  question  asks 

•  Will  there  be  statistically  significant  differences 
between  adjudicators'  perceptions  or  feeling  towards  the 
field  agents  who  wrote  a  high- impact  ROI  as  compared  to 
those  who  wrote  the  low-impact  ROI? 

The  answer  to  that  question  is  given  in  Table  5-1.  Out  of 

fifteen  bipolar  descriptive  adjectives,  only  two  were 

significant  at  5%  level,  while  four  were  significant  at  the 

10%  level. 

1.  Questions  Statistically  Significant  Within  the  5% 
Confidence  Interval 

a.  Question  8:  Agent  Clarity  of  Thinking 

This  bipolar  adjective  determined  whether 
adjudicators  perceive  differences  in  the  clarity  of  thinking 
between  the  field  agent  who  wrote  the  high  or  low-impact  case 
treatments.  The  high- impact  mean  score  (1.9)  falls  just  to 
the  left  of  the  somewhat  clear  thinking  rating,  while  the  low- 
impact  score  (2.25)  lies  between  the  somewhat  clear  thinking 
and  the  neutral  rating.  The  results  indicate  that  the 
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TABLE  5-1 


SUMMARY  OF  QUESTIONNAIRE  PART  III  RESULTS 
ANALYSIS  OF  BOTH  CZARNEK  &  ROKITKA  CASES  (n  =  40) 


ADJUDIC 

:AT0R'S  PERCEPTIO^ 

OF  FIELD  AGENT  WRITING  RC 

)I 

QUEST 
ION  # 

HI 

MEAN 

HI  S.D. 

LI 

MEAN 

LI  S.D. 

F-STAT 

SIG. 

Of  F 

1 

1.78 

0.97 

2.15 

1.03 

1.7 

0.197 

2 

1.79 

0.92 

2.03 

1 

0.67 

0.417 

3 

1.54 

0.72 

1.78 

0.8 

1.35 

0.249 

4 

1.93 

0.97 

2.08 

0.92 

0.088 

0.768 

5 

1.97 

1.12 

2.4 

1.1 

2.78 

0.1 

6 

1.98 

1.07 

2.33 

0.96 

3.29 

0.074 

7 

1.83 

0.84 

2.21 

0.89 

3.81 

0.055 

8 

0.99 

2.25 

0.9 

5.02 

0,028 

9 

2 

1.24 

2.27 

1.11 

1.13 

0.292 

10 

BilSI 

1.01 

2.3 

0.94 

4.92 

0.03 

11 

1.77 

0.99 

2 

0.79 

0.944 

0.335 

12 

1.74 

0.91 

2.05 

0.89 

1.01 

0.318 

13 

1.88 

1.04 

2.15 

1.02 

0.68 

0.413 

14 

1.74 

0.94 

2.23 

1 

3.53 

0.064 

15 

2.15 

1.05 

2.28 

1.04 

0.025 

0.876 

adjudicators  felt  the  writer  of  the  high- impact  ROI  is  a  more 
lucid  thinker  than  his  low-impact  counterpoint.  This  result 
points  out  that  the  ROI  document  design,  organization  and 
writing  style  projects  a  more  intelligent  image  of  the  writer, 
which  could  increase  the  writer's  credibility  with  the  reader. 

Jb.  Question  10:  Efficiency  of  the  Field  Agent 

This  bipolar  adjective  determined  whether 
adjudicators  perceived  differences  in  efficiency  between  the 
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field  agent  who  wrote  the  high  or  low-impact  cases.  The  high- 
impact  mean  score  (1.85)  falls  just  to  the  left  of  the 
somewhat  efficient  rating,  while  the  low-impact  score  (2.3) 
lies  between  the  somewhat  efficient  and  the  neutral  rating. 

The  results  show  that  adjudicators  perceive  the 
high- impact  writer  as  more  efficient  than  the  low-impact 
author.  This  perception  of  efficiency  could  also  increase  the 
field  agent's  credibility  with  the  adjudicators.  In  other 
words,  the  more  efficient  the  field  agent  seems  to  the 
adjudicators,  the  less  likely  they  will  feel  that  the 
Personnel  Security  Investigation  (PSI)  was  conducted  in  a 
shoddy,  haphazard  manner.  These  results  also  suggest  that 
adjudicators  view  the  dociiment  factors  in  the  high- impact 
style  as  a  more  efficient  means  of  conveying  derogatory 
information  than  the  low- intact  style. 

2.  Questions  Statistically  Significant  Within  the  10% 

Interval 

The  results  to  the  rating  of  the  bipolar  adjectives  5, 
6,  7,  and  14  were  found  to  be  significant  within  the  10% 
confidence  interval.  These  bipolar  adjectives  determined 
whether  the  adjudicators  perceived  differences  in  the  field 
agents  in  the  areas  of  organization,  precision,  care,  and 
j udgement  respectively . 

The  high  and  low- inpact  scores  and  their  relative 
position  on  the  rating  scale  are  as  follows: 
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•  IS:  The  high- impact  score  (1.97)  falls  just  to  the  left 
of  the  somewhat  organized  rating.  The  low-impact  score 
(2.4)  lies  between  the  somewhat  organized  and  neutral 
rating. 

•  The  high- impact  score  (1.98)  falls  just  to  the  left 
of  the  somewhat  precise  rating.  The  low-impact  score 
(2.33  lies  between  the  somewhat  precise  and  neutral 
rating. 

•  M7:  The  high- impact  score  (1.83)  falls  to  the  left  of  the 
somewhat  careful  rating.  The  low-impact  score  (2.21)  lies 
between  the  somewhat  careful  and  neutral  rating.  This 
results  shows  a  significance  almost  within  the  5% 
confidence  Interval  (l.e.  5.5%) 

•  #14 :  The  high- impact  score  (1.74)  falls  to  the  left  of 
the  at  fairly  sound  judgment  rating.  The  low-impact  score 
(2.4)  lies  between  the  fairly  sound  judgment  and  neutral 
rating. 

Although  these  results  are  not  a  conclusive  as  the 
findings  from  in  the  5%  interval,  they  indicate  that  the 
writers  of  high- impact  ROI's  are  regarded  as  more  organized, 
precise,  careful,  and  possess  better  judgment  than  their  low- 
impact  counterparts. 

3.  Summary  of  Both  the  Czamek  and  Rokitka  Case  Results 

When  we  combine  the  results  from  the  rating  of  the  two 
bipolar  adjectives  that  passed  the  5%  confidence  test,  and  the 
four  that  passed  the  10%  test,  it  is  fairly  clear  that 
adjudicators  perceive  the  authors  of  the  high- impact  style 


cases  to 

•  possess 

•  be  more 

•  be  more 

•  be  more 


better  judgment 
organized, 
precise, 
careful , 
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•  be  more  clear  thinking, 

•  and  be  more  efficient, 

than  the  writers  of  low-impact  style  cases.  These  perceived 
attributes  may  increase  the  field  agents 's  credibility  in  the 
eyes  of  the  adjudicators  and  thus  produce  greater  confidence 
in  the  ROI  case  information.  This  increased  credibility  and 
confidence  in  the  case  information  would  reduce  the  requests 
for  additional  information  concerning  the  case  stibject. 
Taking  that  argument  one  step  further,  we  could  say  that 
credibility  improves  the  efficient  flow  of  information  in  the 
adjudication  process  and  will  thus  decrease  case  processing 
time. 

As  will  be  seen  in  the  following  sections,  these 
differences  in  perception  are  not  necessarily  true  when  the 
Czarnek  and  Rokitka  cases  or  the  adjudications  sites  are 
analyzed  separately. 

C.  ANALYSIS  OF  THE  HIGH  AND  LOW-IMPACT  CZASNEK  CASES 

The  analysis  results  of  questionnaire  Part  III  using  just 
the  high  and  low-iii5)act  Czarnek  cases  are  presented  in  Table 
5-2.  The  analysis  of  the  fifteen  ratings  of  the  bipolar 
adjectives  showed  that  none  of  the  results  were  found  to  be 
statistically  significant  within  either  the  5%  or  10% 
confidence  intervals. 

These  result  tells  us  that  there  was  no  difference  in 
adjudicators'  perception  of  the  field  agents  who  wrote  either 
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TABLE  5-2 


SUMMARY  OF  QUESTIONNAIRE 
ANALYSIS  OF  THE  CZARNEK 
ADJUDICATOR'S  PERCEPTION 

PART  II 
CASE 

OF  FIEI 

I  RESULTS 

JD  AGENT  V 

FRITING  ROI 

QUEST 
ION  # 

HI 

MEAN 

HI  S.D. 

LI 

MEAN 

LI  S.D. 

F-STAT 

SIG. 
of  F 

1 

1.85 

0.93 

2.3 

1.03 

0.846 

0.364 

2 

1.95 

1.08 

2 

0.97 

0.03 

0.863 

3 

1.58 

0.77 

1.75 

0.85 

0.144 

0.706 

4 

2.1 

1.07 

2.1 

1.07 

0.308 

0.582 

5 

2.12 

1.11 

2.3 

0.98 

0.282 

0.599 

6 

2.1 

1.21 

2.35 

0.81 

1.287 

0.264 

7 

1.85 

0.88 

2.21 

0.71 

1.459 

0.235 

8 

2.16 

1.12 

2.25 

0.85 

0.706 

0.407 

9 

2 

1.26 

2.2 

0.95 

0.172 

0.681 

10 

1.89 

1.05 

2.3 

0.92 

1.743 

0.195 

11 

2 

1.15 

2.15 

0.74 

0.085 

0.772 

12 

1.95 

1.03 

2.11 

0.88 

0.021 

0.885 

13 

2.05 

1.23 

2.2 

0.77 

0.007 

0.935 

14 

1.89 

1.1 

2.25 

0.97 

0.843 

0.365 

15 

2.3 

1.08 

2.25 

0.79 

0.1 

0.754 

the  high  or  low- inpact  Czamek  cases.  We  can  also  infer  that 
these  results  are  not  the  driving  force  behind  the 
statistically  significant  results  to  the  bipolar  ratings  when 
both  the  Czarnek  and  Rokitka  Cases  were  analyzed  together. 
Similar  to  the  Questionnaire  Part  II  results,  this  lack  of 
significant  difference  is  unexpected  because  Czarnek  case 
content  is  much  more  complicated  and  longer  than  Rokitka  case 
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content.  One  would  assume  that  writers  of  high- impact 
documents  containing  longer  and  more  complicated  information 
would  be  more  positively  perceived  than  writer  of  low-impact 
ones. 

D.  AXIALYSIS  OF  THE  HIGH  AND  LOW-IMPACT  ROKITKA  CASES 

The  result  of  Part  III  using  just  the  high  and  low-impact 
Rokitka  case  date  is  presented  in  Table  5-3.  Only  the 
bipolar  rating  for  clarity  of  thinking  (#8)  was  found  to  be 
statistically  significant  within  the  5%  confidence  interval. 
Three  other  bipolar  ratings  of  organization  (#5) ,  efficiency 
(#10)  ,  and  judgment  (#14)  were  significant  within  the  10% 
interval . 

Comparing  these  semantic  differential  results  with  those 
from  our  analysis  of  both  case,  we  see  consistent  findings. 
The  statistically  significant  responses  described  above  were 
also  found  significant  within  either  the  5%  or  10%  confidence 
interval  in  the  combined  Czamek  and  Rokitka  analysis  (Table 
5-1)  . 

Unlike  the  Czamek  cases,  we  can  say  that  the  Rokitka  case 
responses  are  the  driving  force  behind  the  statistically 
significant  results  we  found  when  both  cases  were  combined. 
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TABLE  5-3 


SUMMARY  OF  QUESTIONNAIRE 
ANALYSIS  OF  THE  ROKITKA 
ADJUDICATOR'S  PERCEPTIOK 

PART  II 
CASE 

OF  FIEL 

I  RESULTS 

D  AGENT  V 

rRITING  RC 

II 

QUEST 
ION  # 

HI 

MEAN 

HI  S.D. 

LI 

MEAN 

LI  S.D. 

F-STAT 

SIG. 
of  F 

1 

1.7 

1.03 

2 

1.03 

0.851 

0.362 

2 

1.65 

0.75 

2.05 

1.05 

1.93 

0.173 

3 

1.5 

0.69 

1.8 

0.77 

1.693 

0.201 

4 

1.75 

0.85 

2.05 

0.76 

1.385 

0.247 

5 

1.85 

1.14 

2.5 

1.24 

2.998 

0.091 

6 

1.85 

0.93 

2.32 

1.11 

2.023 

0.163 

7 

1.8 

0.83 

2.2 

1.06 

2.352 

0.134 

a 

x,m 

0,81 

2,25 

0*97 

5*577 

0*024 

9 

2 

1.26 

2.35 

1.27 

1.09 

0.303 

10 

1.8 

1.01 

2.3 

0.98 

3.27 

0.079 

11 

1.55 

0.76 

1.84 

0.83 

1.31 

0.26 

12 

1.55 

0.76 

2 

0.92 

1.884 

0.178 

13 

1.7 

0.8 

2.1 

1.25 

1.089 

0.303 

14 

1.6 

0.75 

2.2 

1.06 

3.219 

0.081 

15 

2 

1.03 

2.3 

1.26 

0.211 

0.648 

E.  ANALYSIS  OF  THE  SEPARATE  ADJUDICATION  SITES  USING  BOTH  THE 
HIGH  AND  LOW-IMPACT  CZARNEK  AND  ROKITKA  CASES 

This  section  presents  and  analyzes  the  Part  III  results 
for  each  separate  adjudication  site.  This  section  will  be 
presented  in  three  subsections:  Army,  Navy  and  DISCR  results. 
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1.  Presentation  and  Analysis  of  the  Army  Adjudication 


Site  Results 

Unlike  the  last  chapter,  the  Army's  questionnaire  Part 
III  results  did  not  dominate  the  other  sites  in  statistically 
significant  results.  Table  5-4  shows  that  the  result  to  the 
judgement  bipolar  rating  (#14)  was  the  only  result  significant 
to  within  either  the  5%  or  10%  confidence  interval.  This 
rating  was  is  consistent  with  the  combined  case  analysis. 


TABLE  5-4 


SUMMARY  OF  QUES 
ANALYSIS  OF  ARJ^ 
ADJUDICATOR'S  E 

TIONNAIRE  PART  III  RESULTS 
[Y  SITE  (n  =  12)  USING  BOTH  CASES 
>ERCEPTION  OF  FIELD  AGENT  WRITING  ROI 

QUEST 
ION  # 

HI 

MEAN 

HI  S.D. 

LI 

MEAN 

LI  S.D. 

F-STAT 

SIG. 
of  F 

1 

2 

1.04 

2.17 

1.03 

0.149 

0.703 

2 

2 

0.85 

2.33 

0.89 

0.889 

0.357 

3 

1.75 

0.97 

2.08 

1.08 

0.597 

0.449 

4 

2.25 

0.75 

2.33 

0.98 

0.052 

0.823 

5 

2.08 

0.79 

2.58 

0.9 

2.093 

0.163 

6 

2.17 

1.03 

2.5 

1.17 

0.986 

0.333 

7 

2.17 

0.94 

2.55 

0.93 

0.86 

0.365 

8 

2 

0.85 

2.42 

1 

2.25 

0.15 

9 

2.67 

1.56 

2.42 

1.24 

0.048 

0.828 

10 

2.33 

1.07 

2.42 

1 

0.239 

0.63 

11 

1.83 

0.94 

2.25 

0.97 

2.05 

0.168 

12 

1.92 

0.9 

2.36 

0.92 

1.66 

0.213 

13 

2 

0.95 

2.33 

1.07 

1.181 

0.291 

14 

1.83 

0.94 

2.42 

1 

3.4 

0.081 

15 

2.08 

^^4^ 

1.08 

1.014 

0.327 

67 


2.  Presentation  euid  Analysis  of  the  Navy  Adjudication 


Site  Results 

The  result  of  the  Navy  site  analysis  are  presented  in 
Table  5-5.  Only  the  "precision"  bipolar  rating  (#6)  was  found 
to  be  statistically  significant  within  the  5%  confidence 
interval.  Three  other  bipolar  ratings  for  clarity  of  thinking 
(#8)  ,  awareness  (#9) ,  and  efficiency  (#10)  were  within  the 
10%  interval.  Comparing  these  results  with  those  from  the 


TABLE  5-5 


SUMMARY  OF  QUESTIONNAIRE  PART  III  RESULTS 

ANALYSIS  OF  NAVY  SITE  (n  =  12)  USING  BOTH  CASES 
ADJUDICATOR'S  PERCEPTION  OF  FIELD  AGENT  WRITING  ROI 

QUEST 
ION  # 

HI 

MEAN 

HI  S.D. 

LI 

MEAN 

LI  S.D. 

F-STAT 

SIG. 

Of  F 

1 

1.67 

0.78 

2.17 

0.72 

1.005 

0.329 

2 

1.82 

0.6 

1.83 

0.72 

0.162 

0.692 

3 

1.55 

0.52 

1.83 

0.58 

0.722 

0.407 

4 

1.75 

0.75 

2.33 

0.98 

0.341 

0.567 

5 

2 

0.82 

2.25 

0.97 

0.278 

0.604 

6 

2.64 

1,03 

5.918 

0.026 

7 

1.67 

0.49 

2 

0.74 

2.033 

0.172 

8 

2 

1 

2.33 

0.98 

3.317 

0.086 

9 

1.67 

0.78 

2.42 

1 

3.938 

0.064 

10 

1.73 

0.65 

2.42 

1.08 

3.938 

0.064 

11 

1.91 

0.94 

2 

0.77 

0.103 

0.752 

12 

1.73 

0.65 

2.08 

o 

• 

VO 

0.054 

0.819 

13 

2 

1.13 

2.25 

1.06 

0.08 

0.781 

14 

1.83 

0.72 

2.33 

1.15 

0.095 

0.761 

15 

£^4^ 

1 

2.75 

^^00^ 

0.937 

analysis  of  both  case,  we  see  consistent  findings,  with  only 


one  exception.  Question  6  was  statistically  significant 
within  the  10%  interval  in  the  previous  analysis,  but  the 
Navy  results  show  it  falling  within  the  5%  interval. 

3.  Presentation  and  Analysis  of  the  DISCR  Adjudication 
Site  Results 

The  DISCR  results  are  presented  in  Table  5-6.  This 
site  produced  no  statistically  significant  results.  Oddly 


TABLE  5-6 


SUMMARY  OF  QUESTIONNAIRE  PART  III  RESULTS 

ANALYSIS  OF  DISCR  SITE  (n  =  12)  USING  BOTH  CASES 
ADJUDICATOR'S  PERCEPTION  OF  FIELD  AGENT  WRITING  ROI 

QUEST 
ION  # 

HI 

MEAN 

HI  S.D. 

LI 

MEAN 

LI  S.D. 

F-STAT 

SIG. 
of  F 

1 

1.67 

1.15 

2.08 

1.44 

0.367 

0.552 

2 

1.58 

1.24 

1.92 

1.38 

0.213 

0.649 

3 

1.33 

0.49 

1.42 

0.67 

0.046 

0.833 

4 

1.58 

1.16 

1.83 

0.94 

0.165 

0.689 

5 

1.82 

1.6 

2.33 

1.5 

0.628 

0.438 

6 

1.67 

1.15 

2 

0.74 

0.667 

0.424 

7 

1.58 

0.9 

2.17 

1.11 

1.815 

0.193 

8 

1.67 

1.15 

2.08 

0.9 

0.969 

0.337 

9 

1.67 

1.15 

2.17 

1.27 

0.978 

0.334 

10 

1.5 

1.17 

2.08 

0.9 

1.788 

0,196 

11 

1.5 

1.17 

1.83 

0.72 

0.407 

0.531 

12 

1.5 

1.17 

1.83 

0.94 

0.338 

0.568 

13 

1.67 

1.15 

1.83 

1.11 

0.032 

0.86 

14 

1.55 

1.21 

2 

0.95 

0.877 

0.361 

15 

1^92 

^^75 

0.86 

0.383 

0.543 

enough,  none  of  the  results  of  the  analysis  were  even  close  to 
falling  within  the  statistically  significant  range  indicating 
there  is  relatively  little  difference  in  DISCR  adjudicators' 
perception  of  the  field  agents  who  wrote  either  the  high  or 
low-impact  styles  cases.  The  reason  why  the  adjudicator's  show 
no  preference  between  the  two  case  may  lie  in  unique 
organizational  norms  at  the  DISCR. 

F.  PRESENTATION  AND  ANALYSIS  OF  THE  MALE  ADJUDICATOR  RESULTS 

Just  as  in  the  previous  chapter,  the  analysis  of  the 
several  demographic  breakdowns  produced  no  significant  results 
except  for  the  male  adjudication  population.  These  result  are 
presented  in  Table  5-7.  The  bipolar  ratings  for  organization 
(#5)  ,  clarity  of  thinking  (#8) ,  and  efficiency  (#10)  were 
found  to  be  statistically  significant  at  the  5%  level,  while 
the  rating  for  thoroughness  (#1),  precision  (#6),  care  (#7), 
confidence  (#11) ,  and  judgement  (#14)  were  statistically 
significant  at  the  10%  level. 

For  the  most  part,  these  results  are  consistent  with  those 
found  in  the  analysis  of  both  cases  using  the  entire 
population  (Table  5-1),  with  three  notable  exceptions.  There 
are  two  additional  bipolar  ratings  (#  1  "thoroughness"  and  #11 
"confidence")  with  statistically  significant  results,  and  one 
question  (#5  "clarity  of  thinking")  with  results  of  greater 
statistical  significance  (i.e.  5%  vice  10%) .  Similar  to  the 
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TABLE  5-7 


SUMMARY  OF  QUESTIONNAIRE 
ANALYSIS  OF  MALE  ADJUDIC 
ADJUDICATOR'S  IMPRESSION 

PART  II  RESULTS 

ATORS  (n  =  9)  USING  BOTH  CASES 
rs  OF  CASE  INFORMATION 

QUEST 
ION  # 

HI 

MEAN 

HI  S.D. 

LI 

MEAN 

LI  S.D. 

F-STAT 

SIG. 
of  F 

1 

1.44 

0.53 

2.22 

1.09 

2.57 

0.094 

2 

1.56 

0.53 

2.11 

1.05 

1.69 

0.215 

3 

1.44 

0.53 

1.67 

0.71 

0.415 

0.53 

4 

1.67 

0.71 

2.44 

1.13 

2.565 

0.132 

l_ 

1.56 

1,17 

0.01 

6 

1.78 

0.67 

2.56 

1.01 

3.181 

0.096 

7 

1.67 

0.5 

2.22 

0.67 

3.845 

0.07 

«... 

1.56 

2.56 

1.01 

5.016 

0,042 

9 

1.56 

0.73 

2.44 

1.24 

2.997 

0.105 

iO . 

1.S6 

0.53 

2.56 

1*01 

5.983 

11 

1.44 

0.53 

2.11 

0.78 

3.756 

0.073 

12 

1.44 

0.53 

1.78 

0.67 

1.143 

0.303 

13 

1.56 

0.53 

2.11 

1.17 

1.307 

0.272 

14 

1.56 

0.53 

2.56 

1.24 

4.302 

0.057 

15 

2 

0.87 

2.11 

1.17 

0.022 

0.889 

male  adjudicator's  result  to  Part  II,  these  results  are 
remarkable  considering  the  small  sample  size  (n=9) . 

6.  CHAPTER  CONCLUSION 

The  result  to  the  combined  Czarnek  and  Rokitka  case 
analysis  point  out  that  there  is  statistically  significant 
evidence  to  show  that  adjudicators  have  different  perceptions 
of  the  field  agents  who  wrote  high- impact  style  cases  versus 
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those  who  composed  low-impact  one.  Specifically,  they  feel 
that  the  agents  writing  in  a  high- impact  style  possess  better 
judgment,  are  better  organized,  and  are  more  precise,  careful, 
clear  thinking,  and  efficient  than  writers  of  the  low-impact 
cases.  These  perceived  attributes  increase  field  agent 
credibility,  which  could  improve  the  efficiency  of  the  entire 
adjudication  process. 

When  the  cases  were  analyzed  independently,  we  found  that 
there  was  1)  no  significant  differences  in  the  adjudicator's 
perception  of  field  agents  who  wrote  the  high  or  low-impact 
Czarnek  cases;  and  2)  statistically  significant  differences  in 
the  perceptions  of  the  high  and  low- intact  Rokitka  writers. 
These  differences  were  found  in  the  areas  of  organization, 
clarity  of  thinking,  efficiency,  and  judgement. 

When  the  analysis  was  broken  down  by  adjudication  site, 
the  statistical  significance  was  not  nearly  as  great  as  the 
results  from  the  entire  population.  This  result  could  be 
attributed  to  the  different  sample  sizes  used  in  the  analysis 
of  both  cases  and  each  site  (i.e.  40  versus  12  adjudicators)  . 
The  greatest  statistical  differences  was  found  in  the  results 
from  the  Navy  adjudicators,  with  the  Army  coming  in  t  distant 
second.  DISCR  came  in  third  with  no  statistically  significant 
results. 

Similar  to  the  result  in  the  previous  chapter,  the  most 
surprising  result  was  found  from  the  analysis  of  the 
demographic  breakdown  of  the  adjudicators.  The  survey 
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responses  from  the  male  adjudicators  produced 
significant  results. 


statistically 


73 


VI.  PRESENTATION  AND  ANALYSIS  OF  PART  IV  RESULTS:  THE 


WRITING  STYLE  PREFERENCES  OF  ADJUDICATORS 

This  chapter  presents  the  results  from  Questionnaire  Part 
IV  (see  APPENDIX  C)  .  Part  IV  tested  the  adjudicators' 
impressions  or  feeling  about  the  high  and  low-impact  writing 
styles  by  having  them  choose  which  case  treatment  they 
prefered  in  relation  to  eight  factors.  These  preference 
factors  include : 

•  presentation  of  information; 

•  ease  of  reading,  remembering,  and  applying  the 
adjudication  criteria; 

•  amount  of  rereading; 

•  confidence  in  adjudication  decision; 

•  competence  of  the  field  agent  writing  the  case; 

•  and  overall  writing  style. 

The  chapter  is  organized  in  five  sections.  The  first 
subsection  gives  direction  on  how  to  read  the  tables  of 
results  and  the  last  draws  conclusions.  The  remaining  middle 
subsections  are  organized  as  follows: 

•  Both  Caaea:  Analysis  of  both  Czarnek  and  Rokitka  high/low 
impact  cases  using  entire  adjudicator  population. 

•  Separate  Adjudication  sites;  Analysis  of  each  Adjudication 
site  (TUnny,  Navy,  and  DISCR)  using  both  cases. 

•  Male  Adjudicators:  Analysis  of  both  cases  using  Male 
Adjudicators. 
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A.  DIRECTION  FOR  READING  TABLES  OF  RESULTS 

There  are  five  tables  of  results  in  this  chapter.  Each 
table  has  a  title  block  and  seven  colmnns  of  information. 
Below  is  an  extract  of  the  title  block  and  column  heading 
with  an  explanation  of  each. 


SUMMARY  OF  QUESTIONNJ 
CASES  AND  ADJUDICATOl 
ADJUDICATOR'S  PREFERI 

MRE  PART  IV  RESULTS 

IS  TESTED  (n  B  sample  size) 

SNCE  FOR  HIGH  OR  LOW-IMPACT  ROI 

QUEST 

# 

PREFER  HI 

PREFER  LI 

CHI-SQR. 

STAT. 

SIG.  OF 

CHI-SQR. 

STAT. 

#  (%) 

#  (%) 

The  title  block  is  the  same  for  all  tables  with  the 
exception  of  the  second  line,  which  is  highlighted  above. 
This  line  states  which  case  and  adjudicators  were  included  in 
the  analysis.  An  explanation  of  the  seven  column  headings 
are  as  follows: 


•  QUEST  #;  The  Part  IV  question  analyzed,  excluding  question 
6  and  8 . 

•  PREFER  HI:  The  number  and  percentage  of  adjudicators  who 
prefer  the  high- impact  style  cases  relative  to  the  tested 
preference  factor. 

•  PREFER  LI:  The  number  and  percentage  of  adjudicators  who 
prefer  the  low-impact  style  cases  relative  to  the  tested 
preference  factor. 

•  CHI-SQR.  STAT. :  The  value  of  the  statistic  resulting  from 
the  Chi -Squared  testing. 

•  SIG.  of  CHI-SQR.  STAT. :  The  significance  of  the  chi-square 
statistic,  which  is  the  probability  that  the  chi-square 
value  is  due  to  a  random  occurrence. 
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The  results  of  questions  which  fall  within  the  5% 
confidence  interval  are  shaded  while,  results  that  meet  the 
10%  interval  are  highlighted  in  bold  type. 

Questions  6  and  8  were  posed  in  a  yes/no  format  and  could 
not  be  statistically  analyzed  using  Chi-squares.  The  answer 
to  these  questions  are  included  in  the  discussion  of  results. 

B.  THE  ANALYSIS  OF  BOTH  THE  CZASNEK  AND  ROKITKA  CASES  RESULTS 

The  third  primary  research  question,  outlined  in  the  first 
chapter,  asks 

•  Will  there  be  statistically  significant  differences  in 
adjudicators'  preferences  for  the  case  information 
presented  in  a  high- intact  versus  the  low-impact  style? 

The  answer  to  that  question  is  given  in  Table  6-1  below: 


TABLE  6-1 


SUMMARY  OF  QUESTIONNAIRE  PART  IV  RESULTS 

ANALYSIS  OF  BOTH  CZARNEK  &  ROKITKA  CASES  (n  =  40) 
ADJUDICATOR'S  PREFERENCE  FOR  HIGH  OR  LOW-IMPACT  ROI 

QUEST 

# 

PREFER  HI 

PREFER  LI 

CHI-SQR. 

STAT. 

SIG.  OF 
CHI-SQR 
STAT. 

# 

{%) 

# 

(%) 

i 

22 

73% 

11 

20% 

0«004 

. 2.  . 

27 

€mr 

33% 

0  *  02S 

3 

2T' 

33% 

0,022 

4 

'  2  ' 

30 

77% 

27 

12 

O.OIS 

7 

6 

50% 

6 

50% 

1 

— 

0*004 

iiiiiiii 

MRMI 

«2%| 

0*015 
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Each  of  the  significant  results  in  the  table,  as  well  as 
the  responses  to  yes/no  questions,  will  be  compared  with 
either  Part  II  or  III  survey  results.  Unless  otherwise  noted, 
only  Part  II  or  III  results  significant  at  the  5%  or  10% 
confidence  levels  are  used  in  the  comparison. 

1.  Question  1:  Presentation  of  Case  Information 

This  question  asks  adjudicators  which  case  treatment 
writing  style  they  prefered.  The  results,  statistically 
significant  at  the  1%  level,  show  that  by  a  29  to  11  margin 
adjudicators  prefer  the  high- impact  style  to  the  low-impact 
one. 

These  results  confirm  the  findings  from  Part  II  of  the 
survey.  Specifically,  comparing  Part  II  high- inpact  and  low- 
impact  results  of  adjudicators'  irtpressions  of  the  way 
derogatory  information  is  presented  reveals  that  the  high- 
impact  style  seems  to 

•  make  it  easier  to  apply  the  adjudication  criteria, 

•  increase  the  adjudicators'  confidence  in  the  quality  of 
their  adjudication  decision, 

•  improve  field  agent  awareness  of  adjudicators'  daily 
reading  requirements, 

•  and  makes  it  easier  to  find  and  remember  derogatory 
information. 

2.  Question  2:  Ease  of  Reading 

This  question  asked  the  adjudicators  which  case  style 
was  easier  to  read.  The  results,  statistically  significant  at 
the  5%  level,  show  that  by  a  27  to  13  margin  adjudicators 
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found  it  easier  to  read  cases  in  the  high-in^jact  rather  than 
the  low-impact  style. 


These  results  directly  verify  findings  of  one  Part  II 
question  and  indirectly  corroborate  the  findings  from  two 
others.  The  directly  applicable  questions  asked  the 
adjudicators  how  easy  it  was  to  read  the  derogatory 
information,  while  the  indirect  questions  determined  to  what 
extent  did  reading  the  cases  caused  fatigue  and  eye  strain. 
The  high- impact  style  made  it  easier  to  read  derogatory 
information  and  reduced  fatigue  and  eye  strain. 

3.  Question  3;  Ease  o£  Remembering  Case  Information 
This  question  asked  adjudicators  if  information  was 

easier  to  remember  with  the  case  treatments  using  either  the 
high  or  low-impact  style.  The  results,  statistically 
significant  at  the  5%  or  10%  level,  show  that  by  a  27  to  13 
margin  adjudicators  more  easily  remembered  case  information 
written  in  the  high- intact  style  than  the  low-impact  one. 

These  results  confirms  the  finding  from  one  Part  II 
question.  This  c[uestion  found  that,  given  the  way  information 
was  presented  in  the  high  and  low-impact  cases,  adjudicators 
felt  that  it  was  easier  to  remember  the  case  information  in 
the  high- impact  style  theui  the  low- inpact  one. 

4.  Question  4i  Frequency  of  Rereading  Case  Information 
This  question  asked  adjudicators  which  case  style  had 

to  be  read  more  often.  The  results,  statistically  significant 
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at  the  1%  level,  show  that  by  a  30  to  9  margin  adjudicators 
had  to  reread  the  case  information  more  often  in  the  low- 
impact  than  the  high- impact  cases. 

These  results  corroborate  the  finding  from  one  Part  II 
question.  This  question  determined  that  adjudicators'  felt 
they  had  to  reread  sentences  more  often  in  the  low-iiiqpact 
style  than  the  high- impact  one. 

5.  Queatlon  5;  Ease  of  Applying  the  Adjudication  Criteria 

This  question  asked  adjudicators  which  writing  style 

made  applying  the  adjudication  criteria  easier.  The  results, 
statistically  significant  at  the  5%  level,  show  that  by  a  27 
to  12  margin  adjudicators  find  applying  the  adjudication 
criteria  easier  in  the  high-irt^jact  style  than  the  low  one. 

These  results  corroborate  the  finding  from  one 
question  in  Part  II.  The  results  to  that  question  indicate 
that  the  adjudicators'  impressions  of  the  way  derogatory 
information  is  presented  in  the  high-iir5>act  style  seems  to 
make  it  easier  to  apply  the  adjudication  criteria  than  in  the 
low-impact  cases. 

6.  Queatlon  6t  Confidence  In  Adjudication  Decision 

This  yes/no  question  directly  asked  adjudicators  if 
the  different  writing  styles  affected  their  confidence  in 
their  adjudication  decision.  By  a  31  to  9  margin  the 
adjudicators  felt  the  different  styles  did  not  affect  their 
confidence. 
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Much  care  must  be  used  when  comparing  these  results 
with  the  findings  from  related  Part  II  questions.  The  key 
obstacle  to  effective  comparison  lies  in  the  fact  that  this 
Part  IV  question  directly  tested  Adjudicators'  confidence  in 
their  adjudication  decision,  while  the  Part  II  questions 
determined  only  indirectly  their  degree  of  confidence.  Part 
II  determined  adjudicators'  relative  confidence  in 

•  the  quality  of  the  field  agent's  investigation, 

•  and  the  soundness  of  their  adjudication  decision. 

These  results  indirectly  indicate  that  the  high- impact  style 
increased  adjudicators'  confidence  in  their  decision  coirpared 
to  the  low-impact  style,  but  the  position  of  the  mean  scores 
show  that  they  felt  relatively  confident  using  either  style. 
Coiribining  the  Part  II  and  IV  results,  we  see  that  indirect 
testing  shows  that  the  high- impact  style  affects  adjudicators' 
confidence  in  both  the  field  agents'  investigation  and  the 
soundness  of  their  adjudication  decision,  though  direct 
testing  does  not. 

However,  we  must  not  ignore  that  24%  of  the 
adjudicators  did  indicate  that  the  different  case  writing 
styles  did  affect  the  confidence  in  their  adjudication 
decision.  The  other  76%  of  adjudicators  may  be  reluctant  to 
say  directly  that  any  case  style  would  affect  their  confidence 
because  this  would  reflect  poorly  on  their  adjudicative 
cdoility. 


80 


7 .  Question  8  and  9 :  Perception  of  the  Field  Agent' s 


Coiq>etence 

Question  #8  was  a  yes/no  question  that  asked 
adjudicators  if  the  different  writing  styles  affected  their 
perception  of  the  field  agents'  competence.  Question  #9  asked 
only  those  who  responded  "yes"  if  the  writer  of  the  high- 
impact  or  the  low-impact  style  was  more  competent. 

Question  #8  results  show  that  one-half  of  the 
adjudicators  (20)  indicated  writing  style  affected  their 
perception  of  field  agent  competence.  The  fact  that  half  of 
the  adjudicators  said  that  style  affected  their  perception  is 
fairly  startling  since  it  indicates  that  style  alone  had  a 
major  impact  on  perception  of  competence. 

The  competence  of  an  individual  is,  however, 
ambiguous,  and  depends  on  the  definition  used.  It  may  be  a 
combination  of  all,  or  part  of  the  descriptive  adjectives  used 
in  Part  III  of  the  survey.  This  study  interprets  con5)etence 
as  the  combination  of  the  descriptive  adjectives  from  part  III 
that  tested  in  the  significant  level. 

With  this  in  mind,  a  comparison  of  the  Part  IV  and  III 
results  reveals  a  pattern  that  supports  the  finding  that  the 
high- impact  style  affects  adjudicators'  perception  of  field 
agents'  con^etence.  Specifically,  Part  III  shows  that 
adjudicators  perceived  field  agents  who  wrote  the  high-in5)act 
cases  to  be  more  organized,  precise,  careful,  clear  thinking, 
and  efficient  than  the  writers  of  the  low-impact  cases.  They 
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also  felt  that  high- impact  case  writers  possessed  better 
judgement.  The  combination  of  these  statistically  significant 
results  points  out  that  writing  style  does  affect 
adjudicators'  perception  of  field  agents'  cort5)etence . 

Futhermore,  the  results  to  Question  #9  show  that  by  a 
margin  of  17  to  4  the  adjudicators  saying  "yes"  to  Question  #8 
feel  that  the  writers  of  the  high-in5)act  cases  are  more 
competent  than  the  writer  that  use  the  low-iir5>act  style.  This 
result  also  indicates  that  the  portion  of  the  adjudicator 
responding  "yes"  feel  that  high- impact  style  reflects  a  more 
competent  writer  than  the  currently  used  low-impact  style. 

8.  Question  10:  Writing  Style  Preference 

This  question  asked  adjudicators  which  style  would 
they  prefer  all  ROI's  be  written.  The  results,  statistically 
significant  at  the  5%  level,  show  that  by  a  margin  of  27  to  12 
adjudicators  prefer  the  high- iit5>act  style  cases  over  the  low- 
impact  style  ones. 

This  result  verifies  the  findings  of  both 
Questionnaire  Part  II  and  III.  Those  survey  results  show  that 
there  is  a  statistically  significant  difference  between  the 
high  and  low-impact  style  cases  in  the  adjudicators' 
impressions  of  case  information  and  their  perceptions  of  field 
agents  writing  the  cases.  Analyzed  together,  the  result  of 
Part  II,  III  and  IV  clearly  point  out  that  the  high- inpact 
style  is  superior  to  the  status  quo  low-impact  style. 
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C.  ANALYSIS  OF  THE  SEPARATE  ADJODICATION  SITE  RESULTS 

This  section  presents  and  analyzes  the  Part  IV  results  for 
each  separate  adjudication  site.  This  subsection  will 
present  qualitative  discussion  of  the  results  from  the  three 
sites:  Army,  Navy  and  DISCR. 

1.  Presentation  and  Analysis  of  the  Army  Adjudication 

Site  Results 

The  Army  Site  showed  the  least  nvimber  of  statistically 
significant  results  to  Questionnaire  Part  IV.  Those  results 
are  presented  in  Table  6-2.  Question  #9,  which  tested  the 
positive  respondents  to  question  #8  (the  Competence  of  field 
agents),  was  significant  at  the  5%  level,  while  Question  #3, 
which  determined  which  case  writing  style  produced  easier  to 
remember  information,  was  significant  at  the  10%  level. 
Although  the  remainder  of  the  results  are  not  significant, 
they  all  favor  the  high- impact  style  documents.  Question  #6 
results  indicate  (4)  yes  and  (8)  no  responses,  while  question 
#8  shows  (5)  yes  and  (7)  no  responses. 

The  only  major  significant  result  showed  by  a  9  to  3 
margin  that  adjudicators  felt  information  presented  in  the 
high- impact  style  was  easier  to  remember  than  that  presented 
in  the  low-impact  style.  Also,  the  5  yes  respondents  to 
question  #8  believe  that  the  high- impact  writers  were  more 
conqpetent  than  the  low-impact  ones. 
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TABLE  6-2 


SUMMARY 

ANALYS] 

ADJUDIC 

r  OF  QUESTIONNAIRE  PART  IV  R] 
:S  OF  ARMY  SITE  (n  =  12)  USIl 
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BOTH  CAS 
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ES 

ZT  ROI 
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# 

PREFER  HI 

PREFER  LI 

CHI-SQR. 

STAT. 
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# 

(%) 

# 
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1 

7 
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5 

42% 
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0.558 

2 

7 

58% 

5 

42% 

0.343 

0.558 

3 

9 

75% 

3 

25% 

3.09 

0.079 

4 

3 

27% 

8 

73% 

2.4 

0.122 

5 

8 

73% 

3 

27% 

2.4 

0.122 

7 

1 

25% 

3 

75% 

NC 

NC 

9 

100% 

0 

in 

5 

0.025 

10 

7 

64% 

4 

36% 

0.782 

0.376 

2.  Presentation  and  Analysis  of  the  Navy  Adjudication 

Site  Results 

The  Navy  Site  showed  the  most  statistically 
significant  responses  to  Questionnaire  Part  IV.  Those 
results  are  presented  in  Table  6-3.  They  show  that  three 
questions  (#'s  1,  2,  and  10)  were  significant  at  the  5% 
confidence  level,  and  question  #2  was  significant  at  the  10% 
level.  The  results  also  indicate  that  question  #6  had  only 
negative  responses,  while  question  #0  had  (4)  positive  and  (8) 
negative  responses. 

The  major  significant  result  showed  that  the 
adjudicators 

•  preferred  the  presentation  in  the  high- intact  style  cases 
(#1)  , 
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•  felt  that  the  information  presented  in  the  high- impact 
style  was  easier  to  read  (#2) , 

•  had  to  reread  information  more  often  in  the  low-impact 
style  cases  (#4) , 

•  and  preferred  that  all  cases  be  written  in  the  high- impact 
style  (#10) . 

Although  the  remainder  of  the  result  are  not  significant,  they 
indicate  that  the  adjudicators  prefer  the  high- impact 
dociiments . 


TABLE  6-3 


SUMMARY  OF  QUESTIONNJ 
ANALYSIS  OF  NAVY  SITl 
ADJUDICATOR'S  PREFER! 

VIRE  PART  IV  RESULTS 
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4 

1 

0% 

11 
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5 

8 
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4 
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7 
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0% 
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0% 

NC 

NC 

9 
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1 

20% 

to 

« 

to 

to 
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10 

— 

5 
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0.045 

3.  Presentation  and  Analysis  of  the  DISCR  Adjudication 
Site  Results 

The  DISCR  Site  showed  the  second  most  statistically 
significant  results  to  Questionnaire  Part  IV.  Those  results 
are  presented  in  Table  6-4.  Although  none  of  the  responses 
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TABLE  6-4 


SUMMARY  OF  QUESTIONNAIRE  PART  IV  RESULTS 

ANALYSIS  OF  DISCR  SITE  (n  =  12)  USING  BOTH  CASES 
ADJUDICATOR'S  PREFERENCE  FOR  HIGH  OR  LOW-IMPACT  ROI 

QUEST 

# 

PREFER  HI 

PREFER  LI 

CHI-SQR. 

STAT. 

SIG.  OF 
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# 

(%) 

# 
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.  1 

9 
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3 
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0.079 

2 

9 
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3 
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0.079 

3 

8 
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4 
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1.33 

0.248 

4 

4 
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8 
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1.33 

0.248 

5 

9 

75% 

3 

25% 

3.09 

0.079 

7 

2 

40% 

3 

60% 

0.139 

0.709 

9 

7 

78% 

2 

22% 

2.72 

0.1 

10 

9 

75% 

3 

25% 

3.09 

0.079 

were  significant  within  the  5%  level,  and  five  questions  (#'s 
1,  2,  5,  9,  and  10  )  were  significant  within  10%  level.  The 
results  also  indicate  that  question  #6  had  (4)  positive  and 
(8)  responses,  while  question  #8  had  (9)  positive  and  (3) 
negative  responses.  The  statistically  significant  results 
showed  that  adjudicators 

•  preferred  the  presentation  in  the  high- impact  style  cases 
(#1) 

•  felt  that  the  information  presented  in  the  high- impact 
style  was  easier  to  read  (#2) 

•  found  it  easier  to  apply  the  adjudication  criteria  to 
information  in  the  high- impact  style  cases  (#5) 

•  and  preferred  that  all  cases  be  written  in  the  high- impact 
style  (#10) . 


Also  the  respondents  who  answered  "yes"  to  question  #8 
indicated  in  question  #9  that  they  believe  the  high- impact 
writers  were  more  competent  than  the  low-impact  ones. 
Although  the  remainder  of  the  results  are  not  statistically 
significant,  they  indicate  that  adjudicators  prefer  the  high- 
impact  documents. 

D.  ANALYSIS  OF  THE  MALE  ADJUDICATOR  RESULTS 

Just  as  in  the  previous  two  chapters,  the  analysis  of  the 
several  demographic  breakdowns  produced  no  significant  results 
except  for  the  male  adjudicator  population.  Those  results  are 
presented  in  Table  6-5.  Seven  preferences  questions  {#'s  1, 
2,  3,  4,  5,  9,  and  10)  were  significant  at  the  5%  level,  and 
none  were  significant  within  10%  level.  The  results  also 
indicate  that  question  #6  had  (1)  positive  and  (8)  negative 
responses,  while  question  #8  had  (5)  positive  and  (4)  negative 
responses.  The  major  significant  result  showed  that  the  male 
adjudicators 

•  preferred  the  presentation  in  the  high- impact  style  cases 
(#1)  , 

•  felt  that  the  information  presented  in  the  high- impact 
style  was  easier  to  read  and  remember  (#'s  2  and  3), 

•  had  to  reread  information  more  often  in  the  low-impact 
style  cases  (#4) , 

•  found  it  easier  to  apply  the  adjudication  criteria  to 
information  in  the  high- impact  style  cases  (#5) , 

•  and  preferred  that  all  cases  be  written  in  the  high- inpact 
style  (#10) . 


87 


TABLE  6-5 


SUMMARY  OF  QUESTIONN; 
ANALYSIS  OF  MALE  ADJX 
ADJUDICATOR'S  PREFERI 

^IRE  PART  IV  R] 
JDICATORS  (n  = 
SNCE  FOR  HIGH  ( 

3SULTS 

9)  USING  BOTH  CASES 

DR  LOW-IMPACT  ROI 

QUEST 

n 

PREFER  HI 

PREFER  LI 

CHI-SQR. 

STAT. 

SIG.  OF 

CHI-SQR. 

STAT. 

# 

{%) 

# 

(%) 

S*7« 

a. 

$ 

1  1 

m 

s 

a 

a9% 

mssmmsssms 

1— 

a 

— 

S.7€ 

7 

0 

0% 

2 

100% 

2 

0.157  1 

9 

0% 

10 

IX% 

S.76  1 

Also  the  respondents  who  answered  "yes"  to  question  #8 
indicated  in  the  question  #9  results  that  they  believe  that 
the  high- intact  writers  were  more  coir^etent  than  the  low- 
in5)act  ones. 

These  results  confirms  the  findings  in  Questionnaire  Part 
II  and  III.  The  male  adjudicators  are  the  mavericks  in  the 
adjudication  population.  They  seem  to  believe  more  strongly 
than  their  female  counterparts  that  the  high- inpact  style 
cases  are  a  great  improvement  over  the  current  low- inpact 
ones. 
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E.  CHAPTER  CGNCLUSION 


The  overall  results  to  Questionnaire  Part  IV  point  out 
that  there  is  statistically  significant  evidence  to  show  that 
adjudicators  prefer  cases  be  written  in  the  high- impact  rather 
than  the  low-impact  style.  Specifically,  the  analysis 
indicates  the  adjudicators  prefer  the  high- impact  over  the 
low-impact  style  relative  to  the  following  factors: 

•  presentation  of  information; 

•  ease  of  reading,  remembering,  and  applying  the 
adjudication  criteria; 

•  amount  of  rereading; 

•  and  overall  writing  style. 

When  the  Questionnaire  Part  IV  analysis  was  broken  down  by 
adjudication  site,  the  statistical  significance  was  not  nearly 
as  great  as  the  results  from  thw  entire  population.  This 
result  could  be  attributed  to  the  different  sanqple  sizes  used 
in  the  analysis  of  both  cases  and  each  site  (i.e.  40  versus  12 
adjudicators)  .  The  greatest  statistical  differences  was  found 
in  the  results  from  the  Navy  adjudicators,  with  DISCR  coming 
second,  and  Army  third. 

Similar  to  the  result  in  the  previous  two  chapters,  the 
most  surprising  result  was  found  from  the  analysis  of  the 
demographic  breakdown  of  the  adjudicators.  The  survey 
responses  from  the  male  adjudicators  produced  statistically 
significant  results. 
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VII.  CONCLnSIQNS  AMD  RECOMMBMDATION 


A.  COMCLUSIOMS 

Examining  Questionnaire  Part  II,  III  and  IV  results  from 
the  high  and  low-impact  treatment  of  both  the  Czamek  and 
Rokitka  cases  point  out  that  the  adjudicators  prefer  the  high- 
iirpact  style  over  the  currently  used  low-impact  one.  The  data 
indicates  the  adjudicators  prefer  the  high- impact  style  for 
its 

•  presentation  of  information; 

•  ease  of  reading,  remembering,  and  applying  the 
adjudication  criteria; 

•  limited  amount  of  rereading; 

•  and  overall  writing  style. 

There  was,  however,  qq  statistical  evidence  to  show  that 
the  high- impact  case  decreases  adjudication  time. 

In  the  area  of  the  adjudicators'  impression  of  the  case 
information,  the  analysis  does  produce  statistically 
significant  evidence  to  indicate  that  adjudicators  feel  that 
the  high- intact  style  is  an  in^jrovement  over  the  low-in^jact 
style.  Specifically,  they  believe  that  the  high- intact  style 

•  provides  better  coverage  of  issues; 

•  makes  it  easier  to  apply  adjudication  criteria  to  the  case 
information; 


•  makes  the  field  agent  who  wrote  the  case  seem  more  aware 
of  1)  how  to  make  the  case  easier  to  read;  2)  their 
informational  needs;  3)  the  extensive  amount  of  their 
daily  reading; 

•  makes  the  field  agent  seem  like  a  better  writer; 

•  makes  the  derogatory  information  in  the  narrative  report 
easier  to  find,  read,  remember  and  adjudicate; 

•  makes  following  the  chronological  sequence  of  events  in 
the  case  easier  to  follow; 

•  and  causes  less  fatigue  while  reading. 

To  a  lesser  degree  they  believe  that  the  high- impact  style 
makes  them  more  confident  in  the  soundness  and  quality  of 
their  adjudication  decision,  causes  them  to  reread  sentences 
less  often,  and  produces  less  eye  stain. 

The  analysis  of  the  adjudicators'  perception  of  the  field 
agents  provided  statistically  significant  evidence  to  show 
that  the  field  agents  who  wrote  high- impact  style  cases  are 
perceived  differently  than  those  who  cortposed  low- inpact  ones. 
Specifically,  they  feel  that  the  agents  writing  in  a  high- 
impact  style  possess  better  judgment,  are  better  organized, 
and  are  more  precise,  careful,  clear  thinking,  and  efficient 
than  writers  of  low-iirpact  cases.  These  perceived  attributes 
may  increase  field  agent  credibility,  which  could  improve  the 
efficiency  of  the  entire  adjudication  process. 

Both  Part  II  and  III  results  indicate  that  there  were 
greater  statistically  significant  differences  between  high  and 
low-inpact  case  treatments  of  the  Rokitka  cases  than  the 
Czamek  cases.  This  result  is  contrary  to  expected  results 
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because  the  Czamek  case  content  is  more  complicated  and 
longer  than  Rokitka  case  content.  One  would  expect  that  the 
longer  and  more  complicated  a  document's  content,  the  more 
helpful  the  high- impact  style  would  be  in  reading,  finding, 
and  processing  information. 

When  the  analysis  was  divided  by  adjudication  site,  the 
statistical  significance  was  not  nearly  as  great  as  the  result 
from  the  entire  population.  However,  this  could  be  attributed 
to  the  smaller  sait^le  sizes  used.  Overall,  the  greatest 
number  of  statistical  differences  were  found  from  the  analysis 
of  the  Navy  adjudication  site  data.  The  Army  adjudication 
site  data  produced  the  second  most  statistically  significant 
result,  while  DISCR  data  came  in  a  distant  third. 

The  most  surprising  result  was  found  when  analyzing  the 
demographic  breakdown  of  adjudicators.  The  male  adjudicators 
found  that  the  high- impact  style  decreased  the  time  required 
to  adjudicate  a  case.  They  also  felt  that  the  high- impact 
cases  were  a  much  greater  iit^jrovement  over  the  current  low- 
impact  cases.  Finally,  their  perception  of  the  field  agents 
who  wrote  the  high-in5)act  cases  was  extremely  different  than 
the  low  impact  authors.  These  results  may  be  due  to  the  fact 
that  males  are  a  minority  and  could  be  considered  outsiders. 
Outsiders  may  be  more  willing  to  change  the  status  quo  of  an 
organization  than  insiders. 
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B.  RECOMMENDATIONS 


The  overall  results  to  Questionnaire  Parts  II,  III,  and  IV 
clearly  point  out  that  the  adjudicators  perceive  the  high- 
impact  style  cases  to  be  superior  to  the  current  low- inpact 
style.  Specifically,  most  adjudicators  prefer  that  all  ROI's 
be  written  in  a  high- impact  style. 

With  this  in  mind,  DIS  should  heed  their  customers' 
desires  and  train  their  field  agent  to  write  ROI's  in  a  high- 
impact  style.  Although  the  high- inpact  style  does  not  at  a 
statistically  significant  level  improve  the  decision  making 
ability  nor  decrease  case  assessment  time,  we  still  believe 
that  DIS  should  adopt  the  high- inpact  style  in  the  spirit  of 
improved  customer  support. 

If  implemented,  the  high- inpact  ROI's  could  increase  the 
efficiency  of  the  adjudication  process.  The  adjudicators  will 
be  seeing  cases  written  in  a  way  they  perceive  as  more 
efficient.  This  perceived  efficiency  may  increase  the  field 
agents 's  credibility  in  the  eyes  of  adjudicators  and  thus 
produce  greater  adjudicator  confidence  in  the  ROI  case 
information.  This  increased  credibility  and  confidence  in  the 
case  information  could  reduce  requests  for  additional 
information  concerning  the  case  subject.  This  reduction  in 
requests  for  additional  information  will  thus  decrease  overall 
case  processing  time. 
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APPENDIX  A 


PART  II 


INSTRUCTIONS 

This  form  is  designed  to  capture  your  impressions  of  the  case  information  you’ve  read 
and  the  adjudicauon  process  you’ve  undergone.  Please  read  each  of  the  following 
statements  carefully,  and  then  place  .an  "X^hext  to  the  place  on  the  scale  that  best 
describes  your  response  to  each  statement. 

For  example,  if  you  were  responding  to  the  statement 

It  was _ for  me  to  find  important  information  in  the  case. 

□  □  □  □  □ 

1  2  3  4  5 

Easy  Difficult 

and  you  felt  that  to  find  information  it  was 

easy:  check  #  1 

somewhat  easy:  check  #  2 

neither  easy 

nor  difficult:  check  #  3 

somewhat  difficult*  check  #  4 

difficult:  check  #  S 


STATEMENTS 

1.  This  case  provided  coverage  of  issues  to  enable  me  to  make  a  high  quality 

adjudication  decision. 

□  □  □  □  □ 

1  2  3  4  S 

Good  Poor 
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2.  The  way  field  agents  presented  derogatory  information  in  the  case  made  it 
to  apply  adjudication  criteria  to  that  information. 

□  □  □  □  □ 


Difficult 


3.  The  way  derogatory  information  was  presented  made  me . 
the  Geld  agents’  investigation. 

□  □  □  □  O 


in  the  quality  of 


GinGdent 


UnconGdent 


4.  After  reading  the  case,  I  am . 


1  2 
GtnGdent 


.  in  the  soundness  of  my  adjudication  decision. 


□  □  □  □  □ 


4  5 

UnconGdent 


S.  The  Geld  agents  seemed  to  be . 
quality  adjudication  decision. 


1  2 
Aware 


,  of  the  information  I  needed  to  make  a  high 


□  □  □  □  □ 


5 

Unaware 


6.  The  Geld  agents  who  wrote  this  case  seemed  to  be _ 

case  easy  for  me  to  r'.ad. 

□  □  □  □  □ 

1  2  3  4  5 

Aware  Unaware 


.  of  how  to  make  the 
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7.  Based  on  the  way  field  agenu  presented  information  in  the  case,  they  seemed  to  be 
_ of  the  large  amounts  of  information  1  have  to  read  daDy. 


□  □ 

1  2 
Aware 


□  □  □ 

3  4  S 

Unaware 


8.  I  believe  the  field  agents  who  wrote  the  narrative  reports  for  this  case  are 
writers. 


1  2  3  4  5 

Good  Poor 


□  □  □  □  □ 


9.  I  _____  that  the  field  agent  who  wrote  the  case’s  subject  interview  section  should 
receive  training  to  make  his/her  writing  more  readable. 


□  □  □ 

1  2  3 

Agree 


□  □ 

4  5 

Disagree 


10.  I  was  able  to  read  this  case 


than  similar  cases. 


□  □ 

1  2 
Faster 


□  □  □ 

3  4  5 

Slower 
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1!.  The  way  field  agenu  presented  information  in  their  narrative  reports  made  it 
_ to  find  derogatory  information. 


□ 

□ 

□ 

□ 

□ 

1 

2 

3 

4  5 

Easy 

Difficult 

12.  Derocatorv  information  written  bv  the  field  aeents  was 

_ to  read.. 

□ 

□ 

□ 

□  , 

□ 

1 

2 

3 

4  S 

Easy 

Difficult 

13.  I  thought  the  subject  interview  section  of  the  case  was _ to  read. 


□  □ 

□ 

□ 

□ 

1  2 

3 

4 

5 

Easy 

Difficult 

14.  The  wav  the  field  agent  presented  derogatory  information  made  it 
rememter  that  information. 

□  □ 

□ 

□ 

□ 

1  2 

3 

4 

5 

Easy 

Difficult 

15.  I  found  the  case  to  be 

to  adjudicate. 

□  □ 

□ 

□ 

□ 

1  2 

3 

4 

5 

Easy 

Difficult 
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16.  It  was _ for  me  to  locate  information  essential  to  my  efficient  adjudication 

of  the  case. 


□  □  □ 

1  2  3 

Easy 


□  □ 

4  5 

Difficult 


17.  To  understand  what  the  field  agent  was  trying  to 

convey,  I _ 

had  to  r-read 

a  section  or  sections  of  the  case. 

□  □  □ 

□ 

□ 

1  2  3 

4 

5 

Often 

Never 

18.  I _ had  to  reread  sentences  to  understand  the  exact  idea  field  agents  were 

trying  to  convey. 


□  □ 

1  2 
Often 


□  □  □ 

3  4  S 

Never 


19.  I  found  it _ to  follow  the  chronological  sequence  of  events  in  the  case. 


□  □  □ 

1  2  3 

Easy 


□  □ 

4  5 

Difficult 


20.  To  help  me  understand  the  logical  connection  benveen  ideas  in  the  case,  I 
_  had  to  either  underline  pans  of  the  case  or  make  notes. 

□  □  □  □  □ 

1  2  3  4  5 

Often  Never 
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21.  I _ encountered  words  that  were  dilficu}t  to  understand  while  reading  the 


Held  agents'  narratives. 

□ 

□ 

□ 

□ 

□ 

1 

2 

3 

4 

5 

Often 

Never 

22.  Reading  this  case  tired  me  out 

□ 

□ 

□ 

□ 

□ 

1 

2 

3 

4 

s 

Agree 

Disagree 

23.  Reading  this  case 

Strained  my  eyes. 

□ 

□ 

□ 

□ 

□ 

1 

2 

3 

4 

5 

Agree 

Disagree 
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APPEKDIZ  B 


PART  Hi 


INSTRUCTIONS 

Please  indicate  your  perceptions  or  feelings  toward  the  field  agents  who  wrote  the  ROI 
you’ve  just  read  by  placing  an  "X"  near  the  item  that  best  describes  that  perception. 

For  example,  if  after  reading  the  RQIr  you  feel  the  field  agents  are 


Thorough: 

check#  1 

Somewhat  thorough: 

check  #  2 

Neither  thorough 
nor  careless: 

check  #  3 

Somewhat  careless: 

check  #  4 

Careless: 

check#S 

Use  this  guideline  to  respond  to  the  following  items. 


PERCEPTIONS  OF  FIELD  AGENTS 


1 

Thorougl 

2.  1 _ 1 

2 

1 

3  4 

1 _ ^ _ 1 

5 

Careless 

1 _ 1 

1  2 
Competent 

3  4  S 

Incompetent 

3.  1  1 

1  1  1 

1  1 

1 

Objective 

2 

3  4 

5 

Biased 
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PERCEPTIONS  OF  FIELD  AGENTS  (Continued) 


4. 

' 

T  2 

Percepuve 

3 

4 

5 

Unperceptive 

5. 

L_  _  1 

I 

I 

I  I 

1  2 
Organi2ed 

3 

4 

5 

Disorganized 

6. 

1  1  1 

I  I 

1  2 
Precise 

3 

4 

5 

Imprecise 

7. 

1.  1  1 

I  I 

I  I 

1  2 
Careful 

3 

4 

5 

Careless 

8. 

1  1  i 

I 

I  I 

* 

1  2 
Qear  thinking 

3 

4 

5 

Fuzsy  thinking 

9. 

1  1 

I 

I  I 

1  2 
Observant 

3 

4 

5 

Unaware 
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PERCEPTIONS  OF  FIELD  AGENTS  (Continued) 


1 

Efficient 

2 

3 

4 

5 

Inefficient 

11. 

1  __  J 

1 

1 

1  J 

1 

Confident 

2 

3 

4 

5 

Unconfident 

12. 

1  1 

1 

1  -  1 

1 

Reliable 

2 

3 

4 

5 

Unreliable 

13. 

1  1 

1 

1  1 

1 

Experienced 

2 

3 

4 

5 

Inexperienced 

14. 

1 _ I 

_ 1 

1  1 

1  2 
Sound  judgment 

3 

4 

5 

Poor  judgment 

15 

L  .  i 

1 

1  1 

1 

Veteran 

2 

3 

4 

5 

Roolde 
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APPENDIX  C 


PART  IV 


Information  presented  in  both  cases  was  written  and  formatted  in  different  styles.  Please 
provide  us  with  your  impressiom  or  feelings  about  these  different  written  styles  by  checking 
the  appropriate  responses  to  the  statements  listed  below. 


1.  I  preferred  the  way  irtformation  wasjprmnted  in 

Case  I  _ 

Case  II _ 

1  I  felt  the  case  that  was  easier  to  read  was 

Case  I  _ 

Caseff _ 

3.  Information  was  easier  to  remember  in 

Casel  _ 

CaseU _ 

4.  I  had  to  reread  information  more  often  in 

Case  I  _ 

CaseU _ 


5. 


I  found  it  easier  to  apply  adjudication  criteria  to  information  in 


6.  I  felt  the  two  cases*  different  writing  styles  affected  my  confidence  in  my  adjudication 
decision. 

Yes  _ 

No  _ 

7.  If  you  answered  "yes*  to  question  6,  did  you  feel  more  confident  of  your  decision  when 
you  adjudicated  Case  I  or  Case  II? 

Casel  _ 

CaseU _ 

8.  I  fell  the  cases’  different  written  communication  styles  affected  my  perception  of  the 
competence  of  the  field  agents. 

Yes  _ 

No  _ 

9.  If  you  answered  ’Ves"  to  question  8,  did  you  feel  that  the  field  agenu  who  wrote  Case 
I  or  Case  II  were  more  competent? 

Casel  _ 

Casell _ 

10.  I  would  prefer  that  gU  ROIs  be  written  in  the  style  of 

Case  I  _ 

Case  n _ 
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